Oct 30 21:13:08.315549 ip-10-0-142-25 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:13:08.742249 ip-10-0-142-25 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:13:08.742249 ip-10-0-142-25 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:13:08.742249 ip-10-0-142-25 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:13:08.742249 ip-10-0-142-25 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:13:08.745893 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745812    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.745893 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745894    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745898    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745901    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745904    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745907    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745909    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745912    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745915    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745918    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745920    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745923    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745925    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745928    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745931    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745934    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745937    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745940    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745942    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745945    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.745961 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745947    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745950    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745953    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745955    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745958    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745961    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745973    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745976    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745978    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745981    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745984    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745986    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745989    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745991    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745994    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745996    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.745999    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746001    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746004    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746006    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.746415 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746009    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746011    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746014    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746016    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746019    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746021    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746024    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746026    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746029    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746031    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746033    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746036    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746038    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746041    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746044    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746047    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746050    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746053    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746056    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.746886 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746059    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746062    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746067    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746071    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746074    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746077    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746080    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746082    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746085    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746087    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746090    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746092    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746095    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746098    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746100    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746103    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746105    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746108    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746110    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.747394 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746113    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746116    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746118    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746121    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746123    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746126    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746128    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746131    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746134    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746137    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746140    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746143    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746147    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746150    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746154    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746157    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746161    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746164    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746167    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746169    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.747854 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746172    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746175    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746177    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746181    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746184    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746186    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746189    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.746191    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747015    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747023    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747026    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747029    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747032    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747034    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747037    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747040    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747042    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747045    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747047    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.748559 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747050    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747054    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747057    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747060    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747062    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747065    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747067    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747070    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747073    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747076    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747078    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747081    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747083    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747088    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747092    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747096    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747100    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747103    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747106    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747108    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.749372 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747111    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747114    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747116    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747119    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747122    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747125    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747127    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747130    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747132    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747135    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747138    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747141    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747144    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747148    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747151    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747153    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747156    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747159    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747161    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747164    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.749848 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747167    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747170    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747172    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747175    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747178    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747180    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747183    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747185    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747188    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747190    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747193    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747195    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747198    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747200    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747203    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747205    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747209    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747212    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747215    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.750337 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747217    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747220    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747222    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747225    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747228    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747231    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747234    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747236    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747239    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747242    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747244    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747246    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747249    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747252    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747255    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747258    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747275    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747278    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747282    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747284    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.750885 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747287    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747289    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747292    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747294    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747297    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747299    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747302    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747304    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747307    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747309    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747312    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747314    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747317    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747319    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747322    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747325    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747864    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747870    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747874    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.751373 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747877    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747880    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747883    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747886    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747888    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747891    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747894    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747897    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747936    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.747993    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748034    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748040    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748044    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748049    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748054    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748059    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748063    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748067    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748072    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748076    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.751825 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748081    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748091    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748095    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748100    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748104    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748109    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748113    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748117    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748122    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748126    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748131    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748135    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748140    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748144    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748154    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748158    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748162    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748167    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748172    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748176    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.752308 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748181    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748636    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748647    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748652    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748660    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748665    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748675    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748679    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748684    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748688    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748692    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748697    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748701    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748705    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748710    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748714    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748718    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748723    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748727    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748736    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.752791 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748741    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748745    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748750    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748756    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748762    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748768    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748773    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748778    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748783    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748787    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748791    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748796    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748805    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748809    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748814    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748819    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748823    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748827    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748831    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.753256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748836    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748840    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748844    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748849    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748853    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748863    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748867    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748871    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748875    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748880    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748884    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748888    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748892    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748896    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748901    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748905    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748909    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748913    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748922    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748926    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.753706 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748930    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748939    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748944    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.748949    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749145    2574 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749156    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749169    2574 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749176    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749184    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749189    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749201    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749207    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749213    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749218    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749224    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749230    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749235    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749242    2574 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749247    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749289    2574 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749295    2574 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749300    2574 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749438    2574 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:13:08.754183 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749448    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749453    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749458    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749464    2574 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749481    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749486    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749491    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749496    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749501    2574 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749506    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749510    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749515    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749519    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749527    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749532    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749538    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749543    2574 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749548    2574 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749555    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749563    2574 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749568    2574 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749573    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749577    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749582    2574 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:13:08.754739 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749588    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749593    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749600    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749605    2574 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749610    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749614    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749619    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749624    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749629    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749633    2574 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749637    2574 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749643    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749648    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749653    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749658    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749663    2574 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749666    2574 flags.go:64] FLAG: --help="false"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749669    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749672    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749676    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749679    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749682    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749686    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:13:08.755333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749692    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749695    2574 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749698    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749702    2574 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749705    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749708    2574 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749711    2574 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749714    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749717    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749720    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749723    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749727    2574 flags.go:64] FLAG: --lock-file=""
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749730    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749734    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749737    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749743    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749746    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749749    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749752    2574 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749755    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749758    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749761    2574 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749764    2574 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749769    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749772    2574 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:13:08.755881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749777    2574 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749780    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749783    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749786    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749788    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749792    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749795    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749800    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749809    2574 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749812    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749815    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749818    2574 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749821    2574 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749824    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749827    2574 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749830    2574 flags.go:64] FLAG: --port="10250"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749833    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749836    2574 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0f43ad8e9cae1c23f"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749842    2574 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749845    2574 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749848    2574 flags.go:64] FLAG: --register-node="true"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749851    2574 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749855    2574 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749858    2574 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749861    2574 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749864    2574 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:13:08.756633 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749868    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749871    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749874    2574 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749878    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749881    2574 flags.go:64] FLAG: --runonce="false"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749885    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749888    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749891    2574 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749894    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749897    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749900    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749903    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749906    2574 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749910    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749913    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749916    2574 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749919    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749922    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749925    2574 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749928    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749934    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749938    2574 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749941    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749971    2574 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749975    2574 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:13:08.757241 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749978    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749981    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749984    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749987    2574 flags.go:64] FLAG: --v="2"
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749992    2574 flags.go:64] FLAG: --version="false"
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.749997    2574 flags.go:64] FLAG: --vmodule=""
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.750001    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.750004    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750137    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750141    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750144    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750148    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750151    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750153    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750156    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750159    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750162    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750165    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750167    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750170    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750173    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750176    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.757832 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750179    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750181    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750184    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750186    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750189    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750192    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750195    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750198    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750200    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750204    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750209    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750212    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750215    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750218    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750221    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750224    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750227    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750230    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750233    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.758363 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750235    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750238    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750241    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750244    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750246    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750249    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750252    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750254    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750257    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750259    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750280    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750286    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750290    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750294    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750297    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750300    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750303    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750307    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750311    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.758813 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750314    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750317    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750319    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750322    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750325    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750328    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750330    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750333    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750336    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750339    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750342    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750344    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750347    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750350    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750353    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750355    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750358    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750361    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750364    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750367    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.759256 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750369    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750372    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750375    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750377    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750381    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750384    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750386    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750389    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750391    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750394    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750396    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750399    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750401    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750404    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750407    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750409    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750412    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750415    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750417    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750420    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.759763 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750422    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750425    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750428    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750430    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750433    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750435    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750438    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750440    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750443    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750446    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750449    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750451    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750454    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750456    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.750484    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750520    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.760244 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750525    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750530    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750533    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750535    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750539    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750542    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750544    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750547    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750551    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750553    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750556    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750560    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750563    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750565    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750568    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750571    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750573    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750576    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750578    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.760691 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750581    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750584    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750586    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750589    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750591    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750594    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750596    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750599    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750602    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750605    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750608    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750611    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750613    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750616    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750621    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750625    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750628    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750631    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750634    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750636    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.761158 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750639    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750642    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750644    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750647    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750651    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750653    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750656    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750658    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750661    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750663    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750666    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750669    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750671    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750673    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750676    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750678    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750681    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750683    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750686    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.761655 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750688    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750691    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750695    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750700    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750703    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750706    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750708    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750712    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750715    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750718    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750720    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750723    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750725    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750728    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750731    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750734    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750736    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750740    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750742    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.762115 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750745    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750747    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750750    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750752    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750755    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750757    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750760    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750763    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750765    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750767    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750770    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750773    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750775    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750778    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750780    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750783    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750786    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750788    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750791    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750794    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.762567 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750798    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750800    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750803    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750805    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750808    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750810    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750813    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.750816    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.757556    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.757572    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.757588    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.757592    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757665    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757671    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757674    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757677    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757679    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757682    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757685    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.763033 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757688    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757690    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757693    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757696    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757699    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757701    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757704    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757707    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757709    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757712    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757715    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757718    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757720    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757723    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757725    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757728    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757731    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757733    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757736    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.763506 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757739    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757742    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757745    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757748    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757751    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757754    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757757    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757760    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757762    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757765    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757767    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757771    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757773    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757776    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757778    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757781    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757783    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757786    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757789    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757792    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.763962 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757794    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757797    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757799    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757802    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757805    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757807    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757810    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757812    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757815    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757817    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757821    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757825    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757828    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757831    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757834    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757837    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757841    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757844    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757847    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757850    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.764479 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757852    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757855    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757857    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757860    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757863    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757865    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757868    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757870    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757873    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757875    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757878    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757881    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757883    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757886    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757888    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757891    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757894    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757896    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757899    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757901    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.764983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757904    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757906    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757909    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757912    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757914    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757917    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757919    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757923    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757927    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757931    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757933    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757936    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757938    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757941    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757943    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757946    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757948    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757951    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757953    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.765470 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.757956    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.757985    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758138    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758142    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758145    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758148    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758151    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758154    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758156    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758159    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758161    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758164    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758168    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758170    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758173    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758175    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.765905 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758178    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758181    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758183    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758186    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758188    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758191    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758193    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758196    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758199    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758202    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758204    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758207    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758209    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758212    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758215    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758218    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758221    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758223    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758226    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.766336 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758228    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758231    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758234    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758236    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758239    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758241    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758244    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758246    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758249    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758252    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758254    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758257    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758260    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758276    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758280    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758282    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758285    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758288    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758290    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.766777 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758293    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758296    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758299    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758301    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758304    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758307    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758310    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758312    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758315    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758317    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758320    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758323    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758325    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758328    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758331    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758333    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758336    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758338    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758341    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758343    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.767214 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758346    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758348    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758351    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758354    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758358    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758362    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758366    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758369    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758372    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758374    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758377    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758379    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758382    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758385    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758387    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758390    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758392    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758395    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758398    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.767693 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758400    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758403    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758405    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758408    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758410    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758413    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758415    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758418    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758420    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758423    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758425    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758428    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758430    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758433    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758435    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.758458    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:13:08.768141 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758491    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758494    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758497    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758500    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758503    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758506    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758509    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758512    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758514    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758517    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758520    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758522    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758525    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758528    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758531    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758533    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758536    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758539    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758541    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:13:08.768544 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758543    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758546    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758549    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758551    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758554    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758557    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758559    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758562    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758564    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758567    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758570    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758572    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758576    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758578    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758581    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758583    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758586    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758588    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758591    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:13:08.768983 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758593    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758596    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758598    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758601    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758603    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758607    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758609    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758611    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758615    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758617    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758620    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758623    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758625    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758627    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758631    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758635    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758639    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758642    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758645    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:13:08.769450 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758648    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758651    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758653    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758657    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758659    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758662    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758664    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758667    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758669    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758672    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758675    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758677    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758680    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758682    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758685    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758687    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758690    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758692    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758695    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758697    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:13:08.769880 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758700    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758702    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758705    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758708    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758711    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758713    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758716    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758718    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758721    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758723    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758726    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758728    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758731    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758733    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758736    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758738    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758741    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758744    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758746    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758748    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:13:08.770416 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758751    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758753    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758756    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758759    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758761    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758764    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758767    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758769    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:08.758772    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.758829    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.760686    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.761601    2574 server.go:1014] "Starting client certificate rotation"
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.761702    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.761726    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.764528    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:13:08.770881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.767291    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:13:08.778053 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.778036    2574 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:13:08.781027 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.781005    2574 log.go:25] "Validated CRI v1 image API"
Oct 30 21:13:08.781795 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.781774    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:13:08.791124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.791102    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a6346f91-3283-40aa-9ded-be1489083030:/dev/nvme0n1p4 d2b4df69-363c-46e0-9b51-e57e7d665682:/dev/nvme0n1p3]
Oct 30 21:13:08.791192 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.791122    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:13:08.797608 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.797498    2574 manager.go:217] Machine: {Timestamp:2026-10-30 21:13:08.797026506 +0000 UTC m=+0.359472345 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099778 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ddd251d4a914d15d0559882a967be SystemUUID:ec2ddd25-1d4a-914d-15d0-559882a967be BootID:1967fb36-9ba8-4672-9851-852186d557d7 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:d3:40:ff:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:d3:40:ff:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:ea:69:53:7c:75 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:13:08.797608 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.797596    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:13:08.797734 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.797701    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:13:08.802409 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.802384    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:13:08.802540 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.802411    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-25.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:13:08.802582 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.802542    2574 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:13:08.802582 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.802550    2574 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:13:08.802582 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.802565    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:13:08.802661 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.802582    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:13:08.803392 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.803382    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:13:08.805978 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.805967    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:13:08.806089 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.806080    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:13:08.806870 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.806861    2574 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:13:08.806910 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.806874    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:13:08.806910 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.806885    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:13:08.806910 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.806894    2574 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:13:08.806910 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.806901    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:13:08.807288 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.807257    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:13:08.810165 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.810148    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:13:08.814111 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.814099    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:13:08.814843 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.814832    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:13:08.814889 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.814861    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:13:08.815008 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.814995    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:13:08.815060 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815014    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:13:08.815060 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815022    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:13:08.815060 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815029    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:13:08.815060 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815038    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:13:08.815060 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815045    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:13:08.815963 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815953    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:13:08.816003 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815966    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:13:08.816003 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815973    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:13:08.816003 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815979    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:13:08.816003 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815987    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:13:08.816003 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.815997    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:13:08.816961 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.816951    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:13:08.816961 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.816961    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:13:08.820755 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.820739    2574 server.go:1263] "Started kubelet"
Oct 30 21:13:08.820945 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.820871    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:13:08.820945 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.820885    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:13:08.821133 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.820976    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:13:08.821576 ip-10-0-142-25 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:13:08.821997 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.821982    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:13:08.825326 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.825310    2574 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:13:08.826488 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:08.826471    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:13:08.829636 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.829622    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:13:08.829636 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.829631    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:13:08.829825 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:08.829780    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:08.830647 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.830630    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:13:08.830749 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.830737    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:13:08.830885 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.830633    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:13:08.830983 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.830970    2574 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:13:08.830983 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.830982    2574 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:13:08.831085 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.831060    2574 factory.go:153] Registering CRI-O factory
Oct 30 21:13:08.831085 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.831074    2574 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:13:08.831182 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.831147    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:13:08.831182 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.831166    2574 factory.go:55] Registering systemd factory
Oct 30 21:13:08.831182 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.831174    2574 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:13:08.831337 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.831190    2574 factory.go:103] Registering Raw factory
Oct 30 21:13:08.831337 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.831228    2574 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:13:08.832194 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.832177    2574 manager.go:319] Starting recovery of all containers
Oct 30 21:13:08.843609 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.843592    2574 manager.go:324] Recovery completed
Oct 30 21:13:08.847575 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.847559    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:08.849203 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.849179    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:08.849314 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.849214    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:08.849314 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.849224    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:08.849759 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.849742    2574 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:13:08.849759 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.849757    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:13:08.849870 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.849746    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:08.849870 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.849776    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:13:08.851592 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.851574    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:08.851679 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.851606    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:08.851679 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.851621    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:08.852052 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.852036    2574 policy_none.go:50] "Start"
Oct 30 21:13:08.852130 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.852061    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:13:08.852130 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.852076    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:13:08.853029 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.853015    2574 policy_none.go:44] "Start"
Oct 30 21:13:08.882099 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.882055    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:13:08.886202 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.886187    2574 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:13:08.886286 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:08.886227    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:13:08.886286 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.886238    2574 server.go:87] "Starting device plugin registration server"
Oct 30 21:13:08.886489 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.886477    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:13:08.886556 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.886533    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:13:08.886649 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.886634    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:13:08.886775 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.886761    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:13:08.886814 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.886777    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:13:08.887252 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:08.887232    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:13:08.887351 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:08.887295    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:08.948388 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.948354    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:13:08.949468 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.949455    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:13:08.949539 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.949475    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:13:08.949539 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.949495    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:13:08.949539 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:08.949525    2574 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:13:08.964346 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.964325    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-25.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:13:08.965235 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:08.964306    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-25.us-west-2.compute.internal.18e36b0f01b0bd62  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-25.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-25.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:13:08.820630882 +0000 UTC m=+0.383076723,LastTimestamp:2026-10-30 21:13:08.820630882 +0000 UTC m=+0.383076723,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-25.us-west-2.compute.internal,}"
Oct 30 21:13:08.986690 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.986669    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:08.987515 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.987497    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:08.987601 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.987527    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:08.987601 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.987555    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:08.987601 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:08.987588    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.010036 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.009951    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-25.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:13:09.019404 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.019374    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-25.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 21:13:09.019517 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.019447    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 21:13:09.019517 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.019403    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-25.us-west-2.compute.internal.18e36b0f0364aed5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-25.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-25.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-25.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:13:08.849200853 +0000 UTC m=+0.411646693,LastTimestamp:2026-10-30 21:13:08.849200853 +0000 UTC m=+0.411646693,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-25.us-west-2.compute.internal,}"
Oct 30 21:13:09.026197 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.026180    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:13:09.026248 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.026218    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ff5pd"
Oct 30 21:13:09.050569 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.050547    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"]
Oct 30 21:13:09.050623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.050602    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:09.051471 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.051457    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:09.051525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.051485    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:09.051525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.051495    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:09.052849 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.052837    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:09.053511 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.053492    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:09.053611 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.053523    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:09.053611 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.053537    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:09.053611 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.053492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.053611 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.053611    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:09.054379 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.054365    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:09.054443 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.054390    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:09.054443 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.054405    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:09.054640 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.054624    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.054707 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.054655    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:09.055373 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.055357    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:09.055462 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.055382    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:09.055462 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.055393    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:09.086561 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.086543    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found" node="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.090456 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.090440    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found" node="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.116676 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.116660    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:13:09.116749 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.116695    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:13:09.125688 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.125668    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.125768 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.125688    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-25.us-west-2.compute.internal\": node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:09.132788 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.132769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.132833 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.132797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.132833 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.132821    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/30e0ff2135e8ccc515bab6d6f7a12d83-config\") pod \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"30e0ff2135e8ccc515bab6d6f7a12d83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.233434 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.233399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.233434 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.233438    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.233624 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.233456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/30e0ff2135e8ccc515bab6d6f7a12d83-config\") pod \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"30e0ff2135e8ccc515bab6d6f7a12d83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.233624 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.233516    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/30e0ff2135e8ccc515bab6d6f7a12d83-config\") pod \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"30e0ff2135e8ccc515bab6d6f7a12d83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.233624 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.233523    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.233624 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.233523    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.241512 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.241497    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:09.342010 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.341945    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:09.389839 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.389820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.393509 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:09.393494    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:09.442829 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:09.442703    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:10.575611 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:10.575594    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:10.676689 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:10.676637    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:10.777592 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:10.777568    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:10.793709 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:10.793690    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:13:10.793886 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:10.793859    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:13:10.793944 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:10.793859    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:13:10.793944 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:10.793836    2574 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aa9979ce62f3e4f9a86916a6635549a1-220db26bc9a348ee.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-142-25.us-west-2.compute.internal.18e36b0f0364f578\": read tcp 10.0.142.25:60828->16.59.72.207:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-142-25.us-west-2.compute.internal.18e36b0f0364f578  default   5276 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-25.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-25.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-25.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 21:13:08 +0000 UTC,LastTimestamp:2026-10-30 21:13:09.051490248 +0000 UTC m=+0.613936088,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-25.us-west-2.compute.internal,}"
Oct 30 21:13:10.878519 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:10.878493    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:10.979180 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:10.979125    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:11.062474 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.062434    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:08:09 +0000 UTC" deadline="2028-08-04 15:23:35.969481064 +0000 UTC"
Oct 30 21:13:11.062474 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.062464    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15450h10m24.907021301s"
Oct 30 21:13:11.079798 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.079775    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:11.180212 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.180181    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:11.196322 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.196303    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:13:11.197298 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.197282    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:13:11.197414 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.197313    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:13:11.197414 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.197328    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:13:11.280821 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.280751    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:11.381865 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.381827    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:11.451918 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:11.451885    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121e9c4b72ca7d02f36ba35f5de497be.slice/crio-94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41 WatchSource:0}: Error finding container 94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41: Status 404 returned error can't find the container with id 94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41
Oct 30 21:13:11.452137 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:11.452106    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e0ff2135e8ccc515bab6d6f7a12d83.slice/crio-d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8 WatchSource:0}: Error finding container d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8: Status 404 returned error can't find the container with id d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8
Oct 30 21:13:11.456014 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.455995    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:13:11.461036 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.461020    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:13:11.482283 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.482263    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:13:11.569364 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.569295    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:13:11.662852 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.662828    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:11.724386 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.724363    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:13:11.725076 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.725063    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:13:11.784090 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.784068    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:13:11.840177 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.840105    2574 apiserver.go:52] "Watching apiserver"
Oct 30 21:13:11.863294 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.863273    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:13:11.944583 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.944557    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:13:11.944878 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.944856    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-vlddw","openshift-dns/node-resolver-z7qz6","openshift-image-registry/node-ca-6prr8","kube-system/global-pull-secret-syncer-26g9z","kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal","openshift-multus/multus-additional-cni-plugins-b94vg","openshift-multus/multus-qxmhx","openshift-multus/network-metrics-daemon-fxrjh","openshift-network-diagnostics/network-check-target-jg5k9","openshift-network-operator/iptables-alerter-blkqp","kube-system/konnectivity-agent-d4w2x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk","openshift-ovn-kubernetes/ovnkube-node-nlctd"]
Oct 30 21:13:11.948292 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.948273    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.950176 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.950157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:11.952135 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.952121    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:11.954101 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.954083    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:11.954201 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.954176    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:11.956184 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.956165    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.958367 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.958334    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.960394 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.960376    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:11.960512 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.960493    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:11.962635 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.962622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.964756 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.964738    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:11.966990 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.966971    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:11.967057 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:11.967036    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:11.969684 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.969663    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:11.972276 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.972260    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:11.974711 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974695    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-hostroot\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.974784 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-kubelet-config\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:11.974784 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974735    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-var-lib-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.974784 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.974891 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974824    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-bin\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.974891 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974845    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-cnibin\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.974891 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cni-binary-copy\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.974891 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974876    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-socket-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.975019 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974918    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-ovn\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975019 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cnibin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.975019 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dfl4\" (UniqueName: \"kubernetes.io/projected/73b98cf0-3a8a-46b3-ac91-f18817011ee8-kube-api-access-4dfl4\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:11.975019 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.974994    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2619e7f0-cd04-4aaf-8543-5db1718c5b57-host-slash\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:11.975171 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975023    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-multus-certs\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.975171 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975043    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12607d3-3f19-4743-807d-8e8356051388-tmp-dir\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:11.975171 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:11.975171 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-log-socket\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975171 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.975171 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975162    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c33c14f-d505-434b-a45d-514cc39522c5-host\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975197    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a12607d3-3f19-4743-807d-8e8356051388-hosts-file\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-systemd-units\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975247    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-netns\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975263    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-device-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-etc-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975305    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-registration-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975332    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-kubelet\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975373    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-script-lib\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975463    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xg7f\" (UniqueName: \"kubernetes.io/projected/2619e7f0-cd04-4aaf-8543-5db1718c5b57-kube-api-access-4xg7f\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:11.975498 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975495    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975518    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-etc-kubernetes\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975540    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975564    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975600    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-config\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-env-overrides\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975686    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2619e7f0-cd04-4aaf-8543-5db1718c5b57-iptables-alerter-script\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975724    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975741    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-netd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975762    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-bin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975776    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975791    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-222vk\" (UniqueName: \"kubernetes.io/projected/f24e3266-f0e8-4610-ab07-0448f4537d25-kube-api-access-222vk\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-systemd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975828    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpz2z\" (UniqueName: \"kubernetes.io/projected/5c33c14f-d505-434b-a45d-514cc39522c5-kube-api-access-wpz2z\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:11.975896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975847    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-netns\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-conf-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clwrj\" (UniqueName: \"kubernetes.io/projected/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-kube-api-access-clwrj\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6d30dd-2479-494f-af1c-dec5361950d9-ovn-node-metrics-cert\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975923    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-daemon-config\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975941    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74q97\" (UniqueName: \"kubernetes.io/projected/a12607d3-3f19-4743-807d-8e8356051388-kube-api-access-74q97\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.975973    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-system-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-k8s-cni-cncf-io\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c33c14f-d505-434b-a45d-514cc39522c5-serviceca\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976126    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-sys-fs\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-node-log\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dr7r\" (UniqueName: \"kubernetes.io/projected/9f6d30dd-2479-494f-af1c-dec5361950d9-kube-api-access-8dr7r\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976248    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-multus\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976269    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-kubelet\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976283    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-dbus\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976301    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-slash\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:11.976531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pvs8\" (UniqueName: \"kubernetes.io/projected/e60d4632-5192-4ce4-9190-36ca9849be8a-kube-api-access-5pvs8\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:11.976957 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.976364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-socket-dir-parent\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:11.985968 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.985927    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"30e0ff2135e8ccc515bab6d6f7a12d83","Type":"ContainerStarted","Data":"d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8"}
Oct 30 21:13:11.987077 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:11.987059    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"121e9c4b72ca7d02f36ba35f5de497be","Type":"ContainerStarted","Data":"94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41"}
Oct 30 21:13:12.001965 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.001948    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qzjzq\""
Oct 30 21:13:12.002090 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.002066    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:13:12.002158 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.002121    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:13:12.002158 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.002144    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:13:12.002261 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.002155    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:13:12.002261 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.002175    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:13:12.003603 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.003582    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:13:12.003724 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.003709    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sqpcg\""
Oct 30 21:13:12.003907 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.003893    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:13:12.003959 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.003918    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:13:12.005095 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.005082    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4v9xp\""
Oct 30 21:13:12.005159 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.005117    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:13:12.005159 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.005146    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:13:12.005232 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.005161    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:13:12.009231 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.009218    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:13:12.009269 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.009221    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:13:12.009355 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.009332    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:13:12.009637 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.009620    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kvf2h\""
Oct 30 21:13:12.009704 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.009653    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:13:12.009704 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.009658    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:13:12.009704 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.009699    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:13:12.010974 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.010960    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:13:12.011296 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.011285    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-79sqh\""
Oct 30 21:13:12.017844 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.017831    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:13:12.018147 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.018134    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:13:12.018212 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.018199    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:13:12.018277 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.018265    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:13:12.018310 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.018270    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x6zwr\""
Oct 30 21:13:12.018310 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.018284    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:13:12.018479 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.018468    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h52wm\""
Oct 30 21:13:12.050307 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.050288    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:13:12.050382 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.050301    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:13:12.063843 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.063826    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:13:12.077073 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-device-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.077144 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077078    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-etc-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077144 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077097    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-kubernetes\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.077144 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077113    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-tmp\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.077144 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce211496-a484-4fc1-bf17-3a926847d855-agent-certs\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:12.077278 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-registration-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.077278 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077161    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-etc-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077278 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077198    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-registration-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.077278 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-kubelet\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077278 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077213    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-device-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.077278 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077278 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077264    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-script-lib\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077246    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-kubelet\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077286    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysconfig\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077322    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077325    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077328    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-systemd\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077382    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xg7f\" (UniqueName: \"kubernetes.io/projected/2619e7f0-cd04-4aaf-8543-5db1718c5b57-kube-api-access-4xg7f\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077421    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-etc-kubernetes\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077467    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077488    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.077525 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077507    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-etc-kubernetes\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077532    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077566    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077609    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-config\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077639    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-env-overrides\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077693    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2619e7f0-cd04-4aaf-8543-5db1718c5b57-iptables-alerter-script\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077710    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077728    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8z9b\" (UniqueName: \"kubernetes.io/projected/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-kube-api-access-k8z9b\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077788    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077849    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-netd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077878    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-netd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-bin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077949    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-bin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-222vk\" (UniqueName: \"kubernetes.io/projected/f24e3266-f0e8-4610-ab07-0448f4537d25-kube-api-access-222vk\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-systemd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078024    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-host\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.078045    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078050    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz2z\" (UniqueName: \"kubernetes.io/projected/5c33c14f-d505-434b-a45d-514cc39522c5-kube-api-access-wpz2z\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078075    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-netns\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078110    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-conf-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078139    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-env-overrides\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078151    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-conf-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.077953    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-script-lib\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.078147    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs podName:73b98cf0-3a8a-46b3-ac91-f18817011ee8 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:12.578109547 +0000 UTC m=+3.108473001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs") pod "network-metrics-daemon-fxrjh" (UID: "73b98cf0-3a8a-46b3-ac91-f18817011ee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078178    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-config\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clwrj\" (UniqueName: \"kubernetes.io/projected/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-kube-api-access-clwrj\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6d30dd-2479-494f-af1c-dec5361950d9-ovn-node-metrics-cert\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.078829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078256    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2619e7f0-cd04-4aaf-8543-5db1718c5b57-iptables-alerter-script\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078278    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-systemd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078281    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-daemon-config\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078308    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74q97\" (UniqueName: \"kubernetes.io/projected/a12607d3-3f19-4743-807d-8e8356051388-kube-api-access-74q97\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078386    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-sys\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078390    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078424    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-system-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-k8s-cni-cncf-io\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078505    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-var-lib-kubelet\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078516    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-k8s-cni-cncf-io\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c33c14f-d505-434b-a45d-514cc39522c5-serviceca\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-sys-fs\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078595    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-system-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078631    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-sys-fs\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-node-log\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078662    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-netns\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dr7r\" (UniqueName: \"kubernetes.io/projected/9f6d30dd-2479-494f-af1c-dec5361950d9-kube-api-access-8dr7r\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.079419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-multus\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078698    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-node-log\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078709    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-kubelet\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-dbus\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078739    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-multus\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078725    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078768    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-kubelet\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078812    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-slash\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078814    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-dbus\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078751    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-slash\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078822    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-daemon-config\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078861    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pvs8\" (UniqueName: \"kubernetes.io/projected/e60d4632-5192-4ce4-9190-36ca9849be8a-kube-api-access-5pvs8\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-modprobe-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078933    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-lib-modules\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-socket-dir-parent\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.078988    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c33c14f-d505-434b-a45d-514cc39522c5-serviceca\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-hostroot\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079030    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-socket-dir-parent\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079039    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-hostroot\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-kubelet-config\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-var-lib-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-kubelet-config\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-bin\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079156    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-var-lib-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079230    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-cnibin\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079252    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-bin\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079294    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-conf\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079315    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-cnibin\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079357    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-run\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079380    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-tuned\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079410    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cni-binary-copy\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-socket-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-ovn\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079526    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cnibin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.080623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079540    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079567    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-ovn\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079584    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cnibin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079592    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-socket-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dfl4\" (UniqueName: \"kubernetes.io/projected/73b98cf0-3a8a-46b3-ac91-f18817011ee8-kube-api-access-4dfl4\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079635    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2619e7f0-cd04-4aaf-8543-5db1718c5b57-host-slash\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079660    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-multus-certs\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079684    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2619e7f0-cd04-4aaf-8543-5db1718c5b57-host-slash\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079688    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12607d3-3f19-4743-807d-8e8356051388-tmp-dir\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079713    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-multus-certs\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-log-socket\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079792    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079808    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cni-binary-copy\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079814    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c33c14f-d505-434b-a45d-514cc39522c5-host\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.079827    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a12607d3-3f19-4743-807d-8e8356051388-hosts-file\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079860    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-systemd-units\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.081067 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079866    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c33c14f-d505-434b-a45d-514cc39522c5-host\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.079875    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret podName:2e048855-3db9-47b4-a1dc-29ee474f05e0 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:12.57986189 +0000 UTC m=+3.110225322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret") pod "global-pull-secret-syncer-26g9z" (UID: "2e048855-3db9-47b4-a1dc-29ee474f05e0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079827    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-log-socket\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079905    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-systemd-units\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079916    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a12607d3-3f19-4743-807d-8e8356051388-hosts-file\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-netns\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079965    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.079993    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-netns\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.080020    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce211496-a484-4fc1-bf17-3a926847d855-konnectivity-ca\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.080891    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12607d3-3f19-4743-807d-8e8356051388-tmp-dir\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.080933    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.081550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.081071    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.082359 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.082329    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6d30dd-2479-494f-af1c-dec5361950d9-ovn-node-metrics-cert\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.100004 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.099961    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:13:12.100004 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.099996    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-b4j77"
Oct 30 21:13:12.102817 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.102802    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-dpfbq\""
Oct 30 21:13:12.102866 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.102803    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ph6zc\""
Oct 30 21:13:12.102866 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.102826    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:13:12.102866 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.102809    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:13:12.139580 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.139556    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-222vk\" (UniqueName: \"kubernetes.io/projected/f24e3266-f0e8-4610-ab07-0448f4537d25-kube-api-access-222vk\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.139869 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.139850    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpz2z\" (UniqueName: \"kubernetes.io/projected/5c33c14f-d505-434b-a45d-514cc39522c5-kube-api-access-wpz2z\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:12.140184 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.140159    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xg7f\" (UniqueName: \"kubernetes.io/projected/2619e7f0-cd04-4aaf-8543-5db1718c5b57-kube-api-access-4xg7f\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:12.140896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.140873    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clwrj\" (UniqueName: \"kubernetes.io/projected/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-kube-api-access-clwrj\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.144542 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.144517    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pvs8\" (UniqueName: \"kubernetes.io/projected/e60d4632-5192-4ce4-9190-36ca9849be8a-kube-api-access-5pvs8\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.144783 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.144768    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74q97\" (UniqueName: \"kubernetes.io/projected/a12607d3-3f19-4743-807d-8e8356051388-kube-api-access-74q97\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:12.144833 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.144811    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dr7r\" (UniqueName: \"kubernetes.io/projected/9f6d30dd-2479-494f-af1c-dec5361950d9-kube-api-access-8dr7r\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.145902 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.145886    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dfl4\" (UniqueName: \"kubernetes.io/projected/73b98cf0-3a8a-46b3-ac91-f18817011ee8-kube-api-access-4dfl4\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:12.171876 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.171860    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:13:12.181286 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-modprobe-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181364 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-lib-modules\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181364 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-conf\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181364 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-run\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181364 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-tuned\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce211496-a484-4fc1-bf17-3a926847d855-konnectivity-ca\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-kubernetes\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181419    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-tmp\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181417    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-run\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181462    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-kubernetes\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181472    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-conf\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181461    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-modprobe-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181533 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181511    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce211496-a484-4fc1-bf17-3a926847d855-agent-certs\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181510    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-lib-modules\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysconfig\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181583    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysconfig\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181598    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-systemd\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181640    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181667    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8z9b\" (UniqueName: \"kubernetes.io/projected/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-kube-api-access-k8z9b\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181720    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-host\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181739    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-systemd\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181751    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.181861 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181832    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-host\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.182224 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-sys\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.182224 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-var-lib-kubelet\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.182224 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce211496-a484-4fc1-bf17-3a926847d855-konnectivity-ca\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:12.182224 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181966    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-sys\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.182224 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.182224 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.181969    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-var-lib-kubelet\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.183503 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.183485    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-tuned\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.183631 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.183615    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-tmp\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.183816 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.183799    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce211496-a484-4fc1-bf17-3a926847d855-agent-certs\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:12.241311 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.241287    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:13:12.241311 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.241309    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:13:12.241311 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.241319    2574 projected.go:196] Error preparing data for projected volume kube-api-access-vc4x2 for pod openshift-network-diagnostics/network-check-target-jg5k9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:12.241491 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.241382    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2 podName:45e2f746-f082-4371-b1c0-375a66cf81da nodeName:}" failed. No retries permitted until 2026-10-30 21:13:12.741367072 +0000 UTC m=+3.271730505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vc4x2" (UniqueName: "kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2") pod "network-check-target-jg5k9" (UID: "45e2f746-f082-4371-b1c0-375a66cf81da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:12.243215 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.243197    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8z9b\" (UniqueName: \"kubernetes.io/projected/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-kube-api-access-k8z9b\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.270795 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.270765    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:13:12.273907 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.273889    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:13:12.278760 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.278741    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:13:12.281658 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.281634    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2619e7f0_cd04_4aaf_8543_5db1718c5b57.slice/crio-8a7e09fcc58f0a0598db52e432a5d10cb378dc4ef12669316847608262c54e92 WatchSource:0}: Error finding container 8a7e09fcc58f0a0598db52e432a5d10cb378dc4ef12669316847608262c54e92: Status 404 returned error can't find the container with id 8a7e09fcc58f0a0598db52e432a5d10cb378dc4ef12669316847608262c54e92
Oct 30 21:13:12.282292 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.282274    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c33c14f_d505_434b_a45d_514cc39522c5.slice/crio-a9e0e935ea6c827ed9a7ac5df6e0c16f0eb8c438dd6bded6ac6de57f6e7dad72 WatchSource:0}: Error finding container a9e0e935ea6c827ed9a7ac5df6e0c16f0eb8c438dd6bded6ac6de57f6e7dad72: Status 404 returned error can't find the container with id a9e0e935ea6c827ed9a7ac5df6e0c16f0eb8c438dd6bded6ac6de57f6e7dad72
Oct 30 21:13:12.287533 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.287516    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode60d4632_5192_4ce4_9190_36ca9849be8a.slice/crio-ae96de185b3a643941046293f739122676d59ab6ab70968dffea95ac225650ec WatchSource:0}: Error finding container ae96de185b3a643941046293f739122676d59ab6ab70968dffea95ac225650ec: Status 404 returned error can't find the container with id ae96de185b3a643941046293f739122676d59ab6ab70968dffea95ac225650ec
Oct 30 21:13:12.289122 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.289107    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qxmhx"
Oct 30 21:13:12.297631 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.297611    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd295e58b_95c4_4dbd_9f04_8c8e2c760dbb.slice/crio-1c65cc69476edcc37bc1a9315675780f94dd8657b3e1e95cc3a2e833f9300904 WatchSource:0}: Error finding container 1c65cc69476edcc37bc1a9315675780f94dd8657b3e1e95cc3a2e833f9300904: Status 404 returned error can't find the container with id 1c65cc69476edcc37bc1a9315675780f94dd8657b3e1e95cc3a2e833f9300904
Oct 30 21:13:12.298884 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.298870    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:13:12.301404 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.301390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:13:12.303598 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.303582    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:12.305578 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.305561    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:13:12.306255 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.306238    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24e3266_f0e8_4610_ab07_0448f4537d25.slice/crio-f2539c35340f8647da0db7956f12ea396b4b245eec3644251d13031915940de4 WatchSource:0}: Error finding container f2539c35340f8647da0db7956f12ea396b4b245eec3644251d13031915940de4: Status 404 returned error can't find the container with id f2539c35340f8647da0db7956f12ea396b4b245eec3644251d13031915940de4
Oct 30 21:13:12.307645 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.307631    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:12.310562 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.310540    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12607d3_3f19_4743_807d_8e8356051388.slice/crio-031196f0b61968e9538c1bc59c11f5c289c07f89daa348feb77d77928fd789c5 WatchSource:0}: Error finding container 031196f0b61968e9538c1bc59c11f5c289c07f89daa348feb77d77928fd789c5: Status 404 returned error can't find the container with id 031196f0b61968e9538c1bc59c11f5c289c07f89daa348feb77d77928fd789c5
Oct 30 21:13:12.313805 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.313782    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6d30dd_2479_494f_af1c_dec5361950d9.slice/crio-cecdfc44da39e62a81290b51a6a4019c80b17bf6496bff3b7780ff1435db71c1 WatchSource:0}: Error finding container cecdfc44da39e62a81290b51a6a4019c80b17bf6496bff3b7780ff1435db71c1: Status 404 returned error can't find the container with id cecdfc44da39e62a81290b51a6a4019c80b17bf6496bff3b7780ff1435db71c1
Oct 30 21:13:12.318476 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:13:12.318458    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d4a4e1b_e451_434f_91f4_b35024c1e4c0.slice/crio-e23af40faa270958303025ff6a162ebb29ec637616758607a885a755a3caf445 WatchSource:0}: Error finding container e23af40faa270958303025ff6a162ebb29ec637616758607a885a755a3caf445: Status 404 returned error can't find the container with id e23af40faa270958303025ff6a162ebb29ec637616758607a885a755a3caf445
Oct 30 21:13:12.397862 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.397804    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:13:12.584137 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.584100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:12.584294 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.584172    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:12.584294 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.584290    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:12.584460 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.584368    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs podName:73b98cf0-3a8a-46b3-ac91-f18817011ee8 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:13.584332549 +0000 UTC m=+4.114695982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs") pod "network-metrics-daemon-fxrjh" (UID: "73b98cf0-3a8a-46b3-ac91-f18817011ee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:12.584460 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.584439    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:12.584578 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.584479    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret podName:2e048855-3db9-47b4-a1dc-29ee474f05e0 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:13.584465518 +0000 UTC m=+4.114828968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret") pod "global-pull-secret-syncer-26g9z" (UID: "2e048855-3db9-47b4-a1dc-29ee474f05e0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:12.785623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.785309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:12.785623 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.785471    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:13:12.785623 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.785490    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:13:12.785623 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.785500    2574 projected.go:196] Error preparing data for projected volume kube-api-access-vc4x2 for pod openshift-network-diagnostics/network-check-target-jg5k9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:12.785623 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:12.785544    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2 podName:45e2f746-f082-4371-b1c0-375a66cf81da nodeName:}" failed. No retries permitted until 2026-10-30 21:13:13.785527712 +0000 UTC m=+4.315891158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vc4x2" (UniqueName: "kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2") pod "network-check-target-jg5k9" (UID: "45e2f746-f082-4371-b1c0-375a66cf81da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:12.989981 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.989944    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"cecdfc44da39e62a81290b51a6a4019c80b17bf6496bff3b7780ff1435db71c1"}
Oct 30 21:13:12.991227 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.991202    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z7qz6" event={"ID":"a12607d3-3f19-4743-807d-8e8356051388","Type":"ContainerStarted","Data":"031196f0b61968e9538c1bc59c11f5c289c07f89daa348feb77d77928fd789c5"}
Oct 30 21:13:12.992464 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.992438    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qxmhx" event={"ID":"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb","Type":"ContainerStarted","Data":"1c65cc69476edcc37bc1a9315675780f94dd8657b3e1e95cc3a2e833f9300904"}
Oct 30 21:13:12.993746 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.993726    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6prr8" event={"ID":"5c33c14f-d505-434b-a45d-514cc39522c5","Type":"ContainerStarted","Data":"a9e0e935ea6c827ed9a7ac5df6e0c16f0eb8c438dd6bded6ac6de57f6e7dad72"}
Oct 30 21:13:12.995091 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.995070    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-blkqp" event={"ID":"2619e7f0-cd04-4aaf-8543-5db1718c5b57","Type":"ContainerStarted","Data":"8a7e09fcc58f0a0598db52e432a5d10cb378dc4ef12669316847608262c54e92"}
Oct 30 21:13:12.996330 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.996305    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d4w2x" event={"ID":"ce211496-a484-4fc1-bf17-3a926847d855","Type":"ContainerStarted","Data":"7a28c672f9841339384d23fcb433779c2d474ac16076e87d31b9117a51aa74f0"}
Oct 30 21:13:12.997690 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.997668    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vlddw" event={"ID":"5d4a4e1b-e451-434f-91f4-b35024c1e4c0","Type":"ContainerStarted","Data":"e23af40faa270958303025ff6a162ebb29ec637616758607a885a755a3caf445"}
Oct 30 21:13:12.998846 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:12.998821    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk" event={"ID":"f24e3266-f0e8-4610-ab07-0448f4537d25","Type":"ContainerStarted","Data":"f2539c35340f8647da0db7956f12ea396b4b245eec3644251d13031915940de4"}
Oct 30 21:13:13.000212 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.000190    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerStarted","Data":"ae96de185b3a643941046293f739122676d59ab6ab70968dffea95ac225650ec"}
Oct 30 21:13:13.104330 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.104241    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:08:11 +0000 UTC" deadline="2028-05-25 05:07:30.031838928 +0000 UTC"
Oct 30 21:13:13.104330 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.104283    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13735h54m16.927560493s"
Oct 30 21:13:13.591651 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.591569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:13.591800 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.591655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:13.591867 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.591818    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:13.591920 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.591879    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs podName:73b98cf0-3a8a-46b3-ac91-f18817011ee8 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:15.591861446 +0000 UTC m=+6.122224886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs") pod "network-metrics-daemon-fxrjh" (UID: "73b98cf0-3a8a-46b3-ac91-f18817011ee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:13.591971 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.591939    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:13.592017 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.591970    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret podName:2e048855-3db9-47b4-a1dc-29ee474f05e0 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:15.591959583 +0000 UTC m=+6.122323016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret") pod "global-pull-secret-syncer-26g9z" (UID: "2e048855-3db9-47b4-a1dc-29ee474f05e0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:13.793642 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.793602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:13.794122 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.793824    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:13:13.794122 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.793842    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:13:13.794122 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.793854    2574 projected.go:196] Error preparing data for projected volume kube-api-access-vc4x2 for pod openshift-network-diagnostics/network-check-target-jg5k9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:13.794122 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.793909    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2 podName:45e2f746-f082-4371-b1c0-375a66cf81da nodeName:}" failed. No retries permitted until 2026-10-30 21:13:15.793891318 +0000 UTC m=+6.324254765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vc4x2" (UniqueName: "kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2") pod "network-check-target-jg5k9" (UID: "45e2f746-f082-4371-b1c0-375a66cf81da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:13.988609 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.988539    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:13.988764 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.988678    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:13.988826 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.988764    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:13.988873 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.988840    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:13.988914 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:13.988896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:13.988978 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:13.988959    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:14.106115 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:14.106061    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:08:11 +0000 UTC" deadline="2028-08-02 21:17:15.16138244 +0000 UTC"
Oct 30 21:13:14.106115 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:14.106111    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15408h4m1.055274516s"
Oct 30 21:13:15.023063 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:15.022446    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"30e0ff2135e8ccc515bab6d6f7a12d83","Type":"ContainerStarted","Data":"eff6cd1f6f82d0d503e8135c02e77c967f300ea21a8e8fc22e170d3974dc8652"}
Oct 30 21:13:15.609900 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:15.609858    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:15.610068 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:15.609943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:15.610138 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.610076    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:15.610190 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.610138    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs podName:73b98cf0-3a8a-46b3-ac91-f18817011ee8 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:19.610119761 +0000 UTC m=+10.140483208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs") pod "network-metrics-daemon-fxrjh" (UID: "73b98cf0-3a8a-46b3-ac91-f18817011ee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:15.610251 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.610203    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:15.610251 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.610239    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret podName:2e048855-3db9-47b4-a1dc-29ee474f05e0 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:19.610227416 +0000 UTC m=+10.140590866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret") pod "global-pull-secret-syncer-26g9z" (UID: "2e048855-3db9-47b4-a1dc-29ee474f05e0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:15.811479 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:15.811058    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:15.811479 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.811217    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:13:15.811479 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.811239    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:13:15.811479 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.811252    2574 projected.go:196] Error preparing data for projected volume kube-api-access-vc4x2 for pod openshift-network-diagnostics/network-check-target-jg5k9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:15.811479 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.811317    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2 podName:45e2f746-f082-4371-b1c0-375a66cf81da nodeName:}" failed. No retries permitted until 2026-10-30 21:13:19.811297245 +0000 UTC m=+10.341660692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vc4x2" (UniqueName: "kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2") pod "network-check-target-jg5k9" (UID: "45e2f746-f082-4371-b1c0-375a66cf81da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:15.984001 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:15.983659    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:15.984001 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.983796    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:15.984001 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:15.983861    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:15.984001 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.983937    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:15.984505 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:15.984358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:15.984505 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:15.984446    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:16.039236 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:16.039099    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerStarted","Data":"1ab00859156a5ebff3e719bc42b4baf5887322fc4f4190d93607e70ff2826bf0"}
Oct 30 21:13:16.045921 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:16.045484    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"121e9c4b72ca7d02f36ba35f5de497be","Type":"ContainerStarted","Data":"dc2dbfb7f92acd1f86e0e19430cc439a62404c68ceb9ab92b9798ef8b91d2b89"}
Oct 30 21:13:16.265035 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:16.264975    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal" podStartSLOduration=5.264955642 podStartE2EDuration="5.264955642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:15.13985324 +0000 UTC m=+5.670216695" watchObservedRunningTime="2026-10-30 21:13:16.264955642 +0000 UTC m=+6.795319099"
Oct 30 21:13:17.051365 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.051254    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d4w2x" event={"ID":"ce211496-a484-4fc1-bf17-3a926847d855","Type":"ContainerStarted","Data":"24bcf675e1c5face2c920983a9b41d72e5998393b4618a0d20dbd11f1fd180b3"}
Oct 30 21:13:17.059257 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.058512    2574 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="1ab00859156a5ebff3e719bc42b4baf5887322fc4f4190d93607e70ff2826bf0" exitCode=0
Oct 30 21:13:17.059257 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.058601    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerDied","Data":"1ab00859156a5ebff3e719bc42b4baf5887322fc4f4190d93607e70ff2826bf0"}
Oct 30 21:13:17.062793 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.062223    2574 generic.go:356] "Generic (PLEG): container finished" podID="121e9c4b72ca7d02f36ba35f5de497be" containerID="dc2dbfb7f92acd1f86e0e19430cc439a62404c68ceb9ab92b9798ef8b91d2b89" exitCode=0
Oct 30 21:13:17.062793 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.062260    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"121e9c4b72ca7d02f36ba35f5de497be","Type":"ContainerDied","Data":"dc2dbfb7f92acd1f86e0e19430cc439a62404c68ceb9ab92b9798ef8b91d2b89"}
Oct 30 21:13:17.184100 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.184041    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d4w2x" podStartSLOduration=2.814232013 podStartE2EDuration="7.184021189s" totalImagesPullingTime="4.369789176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:17.183735851 +0000 UTC m=+7.714099342" watchObservedRunningTime="2026-10-30 21:13:17.184021189 +0000 UTC m=+7.714384647"
Oct 30 21:13:17.577588 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.577538    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:17.578187 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.578156    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:17.984562 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.984485    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:17.984714 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:17.984648    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:17.984778 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.984733    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:17.984831 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:17.984815    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:17.984887 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:17.984875    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:17.984997 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:17.984973    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:18.067594 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:18.066916    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"121e9c4b72ca7d02f36ba35f5de497be","Type":"ContainerStarted","Data":"75f2f2f151b5897effe08f338bc9a9a6cf1f49bc287d1075baf9c799ca39431c"}
Oct 30 21:13:18.070609 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:18.070548    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6prr8" event={"ID":"5c33c14f-d505-434b-a45d-514cc39522c5","Type":"ContainerStarted","Data":"3cd76fa870c67f45aa4624403d446a3bd7ec238aac4984c3bcdf4419dda1747a"}
Oct 30 21:13:18.309822 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:18.309760    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" podStartSLOduration=7.309742507 podStartE2EDuration="7.309742507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:18.185716014 +0000 UTC m=+8.716079470" watchObservedRunningTime="2026-10-30 21:13:18.309742507 +0000 UTC m=+8.840105967"
Oct 30 21:13:18.310993 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:18.310953    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6prr8" podStartSLOduration=2.998959088 podStartE2EDuration="8.31093978s" totalImagesPullingTime="5.311980692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:18.305727891 +0000 UTC m=+8.836091370" watchObservedRunningTime="2026-10-30 21:13:18.31093978 +0000 UTC m=+8.841303236"
Oct 30 21:13:19.075812 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.075781    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk" event={"ID":"f24e3266-f0e8-4610-ab07-0448f4537d25","Type":"ContainerStarted","Data":"2d8cdb553f8533f1b7006710d4ea41c7e22c8a41421d66eddb303aa0da6c3432"}
Oct 30 21:13:19.076230 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.075915    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:13:19.642270 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.642009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:19.642270 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.642094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:19.642270 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.642158    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:19.642270 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.642172    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:19.642270 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.642227    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs podName:73b98cf0-3a8a-46b3-ac91-f18817011ee8 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:27.642208671 +0000 UTC m=+18.172572126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs") pod "network-metrics-daemon-fxrjh" (UID: "73b98cf0-3a8a-46b3-ac91-f18817011ee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:19.642270 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.642248    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret podName:2e048855-3db9-47b4-a1dc-29ee474f05e0 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:27.642237944 +0000 UTC m=+18.172601383 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret") pod "global-pull-secret-syncer-26g9z" (UID: "2e048855-3db9-47b4-a1dc-29ee474f05e0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:19.844165 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.843784    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:19.844165 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.843953    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:13:19.844165 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.843973    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:13:19.844165 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.843985    2574 projected.go:196] Error preparing data for projected volume kube-api-access-vc4x2 for pod openshift-network-diagnostics/network-check-target-jg5k9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:19.844165 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.844045    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2 podName:45e2f746-f082-4371-b1c0-375a66cf81da nodeName:}" failed. No retries permitted until 2026-10-30 21:13:27.844028343 +0000 UTC m=+18.374391780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vc4x2" (UniqueName: "kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2") pod "network-check-target-jg5k9" (UID: "45e2f746-f082-4371-b1c0-375a66cf81da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:19.983969 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.983507    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:19.983969 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.983608    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:19.983969 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.983697    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:19.983969 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.983766    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:19.983969 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:19.983831    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:19.983969 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:19.983910    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:20.516380 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:20.516269    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:13:20.929820 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:20.929656    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:13:20.51629106Z","UUID":"e0d99c06-b32c-4538-9fa0-7dae51ff01e4","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:13:20.931284 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:20.931251    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:13:20.931284 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:20.931286    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:13:21.084327 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.084280    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk" event={"ID":"f24e3266-f0e8-4610-ab07-0448f4537d25","Type":"ContainerStarted","Data":"48d6955dd0bf42a0584b4c4e1721e9a25c4e430f94111ca334164a92a69be25c"}
Oct 30 21:13:21.085747 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.085719    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z7qz6" event={"ID":"a12607d3-3f19-4743-807d-8e8356051388","Type":"ContainerStarted","Data":"0398195641dffac1dfd2d01e921b4dafb98f126fe5c60948f0873631db0bf594"}
Oct 30 21:13:21.090754 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.090727    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qxmhx" event={"ID":"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb","Type":"ContainerStarted","Data":"ed0f66bbb4e7b7770fcdf866639320f1350d68df40fa5d7932206ff16000fcd6"}
Oct 30 21:13:21.204176 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.204128    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-z7qz6" podStartSLOduration=3.136754813 podStartE2EDuration="11.204110459s" totalImagesPullingTime="8.067355646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:21.20371728 +0000 UTC m=+11.734080736" watchObservedRunningTime="2026-10-30 21:13:21.204110459 +0000 UTC m=+11.734473917"
Oct 30 21:13:21.322887 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.322654    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-qxmhx" podStartSLOduration=3.3877792749999998 podStartE2EDuration="11.322636712s" totalImagesPullingTime="7.934857437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:21.322341597 +0000 UTC m=+11.852705052" watchObservedRunningTime="2026-10-30 21:13:21.322636712 +0000 UTC m=+11.853000167"
Oct 30 21:13:21.982793 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.982764    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:21.983181 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:21.982898    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:21.983181 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.982928    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:21.983181 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:21.983002    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:21.983181 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:21.983068    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:21.983181 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:21.983129    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:22.094501 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:22.094471    2574 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="cbb46554dd4c6dee9bcfd65f66e94e3910e5f947e8d37c83aa1e728ccb34dd7a" exitCode=0
Oct 30 21:13:22.094601 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:22.094552    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerDied","Data":"cbb46554dd4c6dee9bcfd65f66e94e3910e5f947e8d37c83aa1e728ccb34dd7a"}
Oct 30 21:13:23.101681 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.101652    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-blkqp" event={"ID":"2619e7f0-cd04-4aaf-8543-5db1718c5b57","Type":"ContainerStarted","Data":"9a1258ad4dfb08c0a35c1282066b08c00f6c259b948454b294dfe81d1ab2ba60"}
Oct 30 21:13:23.103239 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.103213    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vlddw" event={"ID":"5d4a4e1b-e451-434f-91f4-b35024c1e4c0","Type":"ContainerStarted","Data":"2c2e3383736be3a7d078eaf0cfcc46e57562094c09fb2634de7c34c169a582b4"}
Oct 30 21:13:23.108881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.108846    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk" event={"ID":"f24e3266-f0e8-4610-ab07-0448f4537d25","Type":"ContainerStarted","Data":"f5a8184f25f5ed5697ac97c427cdee4f5994edf20cf8d4f2676212c581cec33a"}
Oct 30 21:13:23.218222 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.218163    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-blkqp" podStartSLOduration=4.818446218 podStartE2EDuration="13.218146859s" totalImagesPullingTime="8.399700641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:23.218129295 +0000 UTC m=+13.748492763" watchObservedRunningTime="2026-10-30 21:13:23.218146859 +0000 UTC m=+13.748510316"
Oct 30 21:13:23.337358 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.337309    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk" podStartSLOduration=4.051068991 podStartE2EDuration="13.337294867s" totalImagesPullingTime="9.286225876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:23.337062416 +0000 UTC m=+13.867425870" watchObservedRunningTime="2026-10-30 21:13:23.337294867 +0000 UTC m=+13.867658321"
Oct 30 21:13:23.457083 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.457042    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vlddw" podStartSLOduration=3.25090095 podStartE2EDuration="13.45702837s" totalImagesPullingTime="10.20612742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:23.456860052 +0000 UTC m=+13.987223495" watchObservedRunningTime="2026-10-30 21:13:23.45702837 +0000 UTC m=+13.987391836"
Oct 30 21:13:23.982363 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.982311    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:23.982363 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.982360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:23.982606 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:23.982449    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:23.982606 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:23.982499    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:23.982708 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:23.982602    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:23.982708 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:23.982659    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:24.113195 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:24.113160    2574 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="f5253043fe816611ce6874f00cb04f44cc93c76d2e79684614699a38e970bd74" exitCode=0
Oct 30 21:13:24.113881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:24.113278    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerDied","Data":"f5253043fe816611ce6874f00cb04f44cc93c76d2e79684614699a38e970bd74"}
Oct 30 21:13:24.880785 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:24.880756    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:24.880951 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:24.880867    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:13:24.881715 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:24.881691    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:13:25.983055 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:25.982846    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:25.983595 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:25.982860    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:25.983595 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:25.982893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:25.983595 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:25.983178    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:25.983595 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:25.983259    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:25.983595 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:25.983362    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:26.120308 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:26.120271    2574 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="f05fa59705456101284dc9f5baf0cf3e3c355e4b905638749ee765ac21101a8b" exitCode=0
Oct 30 21:13:26.120308 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:26.120303    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerDied","Data":"f05fa59705456101284dc9f5baf0cf3e3c355e4b905638749ee765ac21101a8b"}
Oct 30 21:13:27.703269 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:27.703235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:27.703680 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:27.703319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:27.703680 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.703486    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:27.703680 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.703534    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs podName:73b98cf0-3a8a-46b3-ac91-f18817011ee8 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:43.703521935 +0000 UTC m=+34.233885368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs") pod "network-metrics-daemon-fxrjh" (UID: "73b98cf0-3a8a-46b3-ac91-f18817011ee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:27.703816 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.703799    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:27.703935 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.703843    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret podName:2e048855-3db9-47b4-a1dc-29ee474f05e0 nodeName:}" failed. No retries permitted until 2026-10-30 21:13:43.703831477 +0000 UTC m=+34.234194910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret") pod "global-pull-secret-syncer-26g9z" (UID: "2e048855-3db9-47b4-a1dc-29ee474f05e0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:27.905233 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:27.905200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:27.905439 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.905412    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:13:27.905556 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.905447    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:13:27.905556 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.905462    2574 projected.go:196] Error preparing data for projected volume kube-api-access-vc4x2 for pod openshift-network-diagnostics/network-check-target-jg5k9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:27.905556 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.905526    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2 podName:45e2f746-f082-4371-b1c0-375a66cf81da nodeName:}" failed. No retries permitted until 2026-10-30 21:13:43.905507598 +0000 UTC m=+34.435871047 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vc4x2" (UniqueName: "kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2") pod "network-check-target-jg5k9" (UID: "45e2f746-f082-4371-b1c0-375a66cf81da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:27.982536 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:27.982465    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:27.982536 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:27.982497    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:27.982715 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:27.982463    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:27.982715 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.982588    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:27.982715 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.982702    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:27.982830 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:27.982785    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:29.130309 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:29.130242    2574 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="23e79ba76906ad165f9a6d9076c5c98edde98cc454549d28398c20718fd42d3f" exitCode=0
Oct 30 21:13:29.130309 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:29.130282    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerDied","Data":"23e79ba76906ad165f9a6d9076c5c98edde98cc454549d28398c20718fd42d3f"}
Oct 30 21:13:29.982990 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:29.982832    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:29.983113 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:29.983051    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:29.983113 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:29.982944    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:29.983113 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:29.983106    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:29.983215 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:29.982911    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:29.983215 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:29.983159    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:30.135134 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.135106    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"1c27dad60cf67ce36fa1c17d30191e01711f527139afd8f61371350ae8f5e5e8"}
Oct 30 21:13:30.135632 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.135175    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"4e665fe395d2891cefbb0b2c0551e5214f8ce1af1ace1b6371b6b3230ad2ba37"}
Oct 30 21:13:30.135632 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.135222    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"e79d44fd1d8bd77a697cecd0c4984436b0f81e450849289943ac0dfebfa308fa"}
Oct 30 21:13:30.135632 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.135256    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"34780a855c283523319b6c376707815c3b845e7bb43656d4fc113095ce684f93"}
Oct 30 21:13:30.135632 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.135286    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"5d7bf1d30d545b775959ff4ae69fbc3a7ea3062016f66e873885857b7a6f6d9b"}
Oct 30 21:13:30.135632 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.135322    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"d027e5044674cde9223fd81d7e562588adb7b08bf839326a642180ce7ddbbe7d"}
Oct 30 21:13:30.138032 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.138009    2574 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="262218a4d13282700bbc586605219999fc5ae084031d2c9342445e77a469c5d4" exitCode=0
Oct 30 21:13:30.138143 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:30.138039    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerDied","Data":"262218a4d13282700bbc586605219999fc5ae084031d2c9342445e77a469c5d4"}
Oct 30 21:13:31.143823 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:31.143785    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b94vg" event={"ID":"e60d4632-5192-4ce4-9190-36ca9849be8a","Type":"ContainerStarted","Data":"b1b5d5cdecf59bed7f5d3edcb717185ca482f64dbec6a7308508b57aeec9e31e"}
Oct 30 21:13:31.267508 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:31.267458    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b94vg" podStartSLOduration=8.364921008 podStartE2EDuration="21.267443081s" totalImagesPullingTime="12.902522073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:31.266953931 +0000 UTC m=+21.797317386" watchObservedRunningTime="2026-10-30 21:13:31.267443081 +0000 UTC m=+21.797806570"
Oct 30 21:13:31.982561 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:31.982530    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:31.982561 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:31.982548    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:31.982811 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:31.982631    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:31.982811 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:31.982654    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:31.982811 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:31.982702    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:31.982811 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:31.982787    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:32.149471 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:32.149379    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"d825c4a2cbe0326aebb4f105c8e081eef3bfcb28128f8a789fc5b7cbe781eccf"}
Oct 30 21:13:33.982299 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:33.982264    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:33.982299 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:33.982291    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:33.982889 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:33.982281    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:33.982889 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:33.982415    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:33.982889 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:33.982513    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:33.982889 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:33.982601    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:34.157022 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:34.156942    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" event={"ID":"9f6d30dd-2479-494f-af1c-dec5361950d9","Type":"ContainerStarted","Data":"8a07f965d5bb10b01e55ae6341e0e4d0c5aacdd3cb3889d669839ad238106074"}
Oct 30 21:13:34.157228 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:34.157212    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:34.172248 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:34.172224    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:34.438869 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:34.438776    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd" podStartSLOduration=7.370968833 podStartE2EDuration="24.438763935s" totalImagesPullingTime="17.067795102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:34.387843284 +0000 UTC m=+24.918206739" watchObservedRunningTime="2026-10-30 21:13:34.438763935 +0000 UTC m=+24.969127410"
Oct 30 21:13:35.159446 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:35.159414    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:35.159841 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:35.159505    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:35.173484 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:35.173456    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:13:35.982015 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:35.981978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:35.982179 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:35.982036    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:35.982179 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:35.982049    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:35.982179 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:35.982122    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:35.982294 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:35.982195    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:35.982337 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:35.982309    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:36.561757 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:36.561721    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-26g9z"]
Oct 30 21:13:36.562230 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:36.561831    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:36.562230 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:36.561922    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:36.564802 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:36.564783    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jg5k9"]
Oct 30 21:13:36.564869 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:36.564860    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:36.564948 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:36.564933    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:36.565275 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:36.565257    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxrjh"]
Oct 30 21:13:36.565357 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:36.565336    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:36.565449 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:36.565431    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:37.981821 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:37.981789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:37.982187 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:37.981902    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:37.982187 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:37.981977    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:37.982187 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:37.982084    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:38.982506 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:38.982475    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:38.983048 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:38.982601    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:39.983077 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:39.983044    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:39.983460 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:39.983134    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:39.983460 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:39.983217    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:39.983460 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:39.983311    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:40.361924 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:40.361892    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d4w2x_ce211496-a484-4fc1-bf17-3a926847d855/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:40.981816 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:40.981786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:40.981985 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:40.981893    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:41.982768 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:41.982736    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:41.983124 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:41.982915    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:41.983124 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:41.982948    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:41.983124 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:41.983025    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:42.903478 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:42.903454    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z7qz6_a12607d3-3f19-4743-807d-8e8356051388/dns-node-resolver/0.log"
Oct 30 21:13:42.982063 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:42.982027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:42.982222 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:42.982146    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:43.498657 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:43.498630    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6prr8_5c33c14f-d505-434b-a45d-514cc39522c5/node-ca/0.log"
Oct 30 21:13:43.710157 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:43.710120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:43.710328 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:43.710174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:43.710328 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.710262    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:43.710328 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.710264    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:43.710328 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.710311    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs podName:73b98cf0-3a8a-46b3-ac91-f18817011ee8 nodeName:}" failed. No retries permitted until 2026-10-30 21:14:15.71029824 +0000 UTC m=+66.240661674 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs") pod "network-metrics-daemon-fxrjh" (UID: "73b98cf0-3a8a-46b3-ac91-f18817011ee8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:13:43.710328 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.710324    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret podName:2e048855-3db9-47b4-a1dc-29ee474f05e0 nodeName:}" failed. No retries permitted until 2026-10-30 21:14:15.710317934 +0000 UTC m=+66.240681367 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret") pod "global-pull-secret-syncer-26g9z" (UID: "2e048855-3db9-47b4-a1dc-29ee474f05e0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:13:43.911133 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:43.911060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:43.911272 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.911190    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:13:43.911272 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.911202    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:13:43.911272 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.911211    2574 projected.go:196] Error preparing data for projected volume kube-api-access-vc4x2 for pod openshift-network-diagnostics/network-check-target-jg5k9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:43.911272 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.911255    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2 podName:45e2f746-f082-4371-b1c0-375a66cf81da nodeName:}" failed. No retries permitted until 2026-10-30 21:14:15.911243617 +0000 UTC m=+66.441607050 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vc4x2" (UniqueName: "kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2") pod "network-check-target-jg5k9" (UID: "45e2f746-f082-4371-b1c0-375a66cf81da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:13:43.982587 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:43.982556    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:43.982741 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.982653    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:43.982741 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:43.982724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:43.982825 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:43.982801    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:44.981719 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:44.981691    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:44.982096 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:44.981799    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:45.982312 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:45.982278    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:45.982712 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:45.982403    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:45.982712 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:45.982460    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:45.982712 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:45.982551    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:46.981778 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:46.981742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:46.981942 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:46.981855    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:47.982513 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:47.982481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:47.982513 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:47.982505    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:47.982925 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:47.982597    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:47.982925 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:47.982716    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:48.982490 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:48.982457    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:48.982681 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:48.982586    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:49.983163 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:49.983129    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:49.983581 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:49.983263    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:49.983581 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:49.983299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:49.983581 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:49.983425    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:50.982561 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:50.982529    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:50.982725 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:50.982638    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:51.982280 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:51.982252    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:51.982668 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:51.982366    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:51.982668 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:51.982424    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:51.982668 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:51.982506    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:52.982582 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:52.982543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:52.982967 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:52.982662    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:53.982653 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:53.982623    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:53.983011 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:53.982660    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:53.983011 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:53.982739    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:53.983011 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:53.982872    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:54.982205 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:54.982013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:54.982402 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:54.982281    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:55.981758 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:55.981721    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:55.982187 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:55.981867    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:55.982187 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:55.981918    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:55.982187 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:55.982036    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:56.982216 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:56.982185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:56.982599 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:56.982286    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:57.982596 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:57.982565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:57.982968 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:57.982661    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26g9z" podUID="2e048855-3db9-47b4-a1dc-29ee474f05e0"
Oct 30 21:13:57.982968 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:57.982739    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:13:57.982968 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:57.982822    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jg5k9" podUID="45e2f746-f082-4371-b1c0-375a66cf81da"
Oct 30 21:13:58.982506 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:58.982468    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:13:58.982701 ip-10-0-142-25 kubenswrapper[2574]: E1030 21:13:58.982603    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxrjh" podUID="73b98cf0-3a8a-46b3-ac91-f18817011ee8"
Oct 30 21:13:59.287385 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.287342    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeReady"
Oct 30 21:13:59.287546 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.287481    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:13:59.455779 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.455747    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8fpfq"]
Oct 30 21:13:59.457430 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.457413    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8vhcn"]
Oct 30 21:13:59.457550 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.457532    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.459051 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.459035    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q2p4x"]
Oct 30 21:13:59.459158 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.459143    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.460598 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.460580    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:13:59.510562 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.510535    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2p4x"]
Oct 30 21:13:59.512972 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.512949    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:13:59.513158 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.513142    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwnd5\""
Oct 30 21:13:59.513764 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.513743    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8vhcn"]
Oct 30 21:13:59.514291 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.514272    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8fpfq"]
Oct 30 21:13:59.515868 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.515841    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:13:59.515954 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.515923    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:13:59.516004 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.515959    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b9xqh\""
Oct 30 21:13:59.516091 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.516079    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:13:59.559445 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.559370    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:13:59.588667 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.588638    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:13:59.588667 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.588644    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:13:59.613540 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.613519    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:13:59.613617 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.613543    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:13:59.613651 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.613610    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-lm9pr\""
Oct 30 21:13:59.617850 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.617828    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.617927 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.617877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/830286ce-0d57-4cb7-8f13-a298b08bd319-metrics-tls\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.617968 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.617943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c54779-d916-481f-b659-55e1b3630b77-cert\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:13:59.618018 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.617965    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/830286ce-0d57-4cb7-8f13-a298b08bd319-config-volume\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.618072 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.618059    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.618108 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.618084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k6cd\" (UniqueName: \"kubernetes.io/projected/c6c54779-d916-481f-b659-55e1b3630b77-kube-api-access-6k6cd\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:13:59.618166 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.618154    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz2dz\" (UniqueName: \"kubernetes.io/projected/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-api-access-jz2dz\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.618214 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.618174    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9nsw\" (UniqueName: \"kubernetes.io/projected/830286ce-0d57-4cb7-8f13-a298b08bd319-kube-api-access-w9nsw\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.618272 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.618251    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-data-volume\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.618449 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.618294    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830286ce-0d57-4cb7-8f13-a298b08bd319-tmp-dir\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.621174 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.618661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-crio-socket\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.719989 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.719954    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k6cd\" (UniqueName: \"kubernetes.io/projected/c6c54779-d916-481f-b659-55e1b3630b77-kube-api-access-6k6cd\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:13:59.720168 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2dz\" (UniqueName: \"kubernetes.io/projected/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-api-access-jz2dz\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.720168 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9nsw\" (UniqueName: \"kubernetes.io/projected/830286ce-0d57-4cb7-8f13-a298b08bd319-kube-api-access-w9nsw\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.720168 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-data-volume\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.720168 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830286ce-0d57-4cb7-8f13-a298b08bd319-tmp-dir\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.720168 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720102    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-crio-socket\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.720168 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720117    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.720168 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720150    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/830286ce-0d57-4cb7-8f13-a298b08bd319-metrics-tls\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.720523 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720177    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c54779-d916-481f-b659-55e1b3630b77-cert\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:13:59.720523 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/830286ce-0d57-4cb7-8f13-a298b08bd319-config-volume\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.720523 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720242    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.720631 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720559    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830286ce-0d57-4cb7-8f13-a298b08bd319-tmp-dir\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.720675 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720621    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-data-volume\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.720946 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720914    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/830286ce-0d57-4cb7-8f13-a298b08bd319-config-volume\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.721059 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720918    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-crio-socket\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.721059 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.720979    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.724115 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.724094    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.724291 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.724271    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/830286ce-0d57-4cb7-8f13-a298b08bd319-metrics-tls\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.724333 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.724292    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c54779-d916-481f-b659-55e1b3630b77-cert\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:13:59.795185 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.795152    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k6cd\" (UniqueName: \"kubernetes.io/projected/c6c54779-d916-481f-b659-55e1b3630b77-kube-api-access-6k6cd\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:13:59.795309 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.795238    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2dz\" (UniqueName: \"kubernetes.io/projected/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-api-access-jz2dz\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:13:59.797930 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.797903    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9nsw\" (UniqueName: \"kubernetes.io/projected/830286ce-0d57-4cb7-8f13-a298b08bd319-kube-api-access-w9nsw\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:13:59.983665 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.983634    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:13:59.984105 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:13:59.983809    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:00.037510 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.037479    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:14:00.037658 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.037535    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:14:00.038025 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.038006    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q4s27\""
Oct 30 21:14:00.038113 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.038098    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:14:00.038178 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.038142    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5fn5j\""
Oct 30 21:14:00.070410 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.070390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:00.076132 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.076112    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:00.080062 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.079714    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:14:00.508640 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:00.508612    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830286ce_0d57_4cb7_8f13_a298b08bd319.slice/crio-53fcd4b90605a624857b017ef226e0d4d3eadd6aaacb51708e0abbb27390ac0e WatchSource:0}: Error finding container 53fcd4b90605a624857b017ef226e0d4d3eadd6aaacb51708e0abbb27390ac0e: Status 404 returned error can't find the container with id 53fcd4b90605a624857b017ef226e0d4d3eadd6aaacb51708e0abbb27390ac0e
Oct 30 21:14:00.509302 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.509277    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8fpfq"]
Oct 30 21:14:00.512357 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.512327    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8vhcn"]
Oct 30 21:14:00.512853 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.512834    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2p4x"]
Oct 30 21:14:00.542855 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:00.542832    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53cd7b3b_32e9_4fed_9b76_a16d9ca69d14.slice/crio-fd642d2cf0fec17fd573570b7c747543289634a87bbe54d3fb539d79fe8dd3e0 WatchSource:0}: Error finding container fd642d2cf0fec17fd573570b7c747543289634a87bbe54d3fb539d79fe8dd3e0: Status 404 returned error can't find the container with id fd642d2cf0fec17fd573570b7c747543289634a87bbe54d3fb539d79fe8dd3e0
Oct 30 21:14:00.543539 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:00.543504    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c54779_d916_481f_b659_55e1b3630b77.slice/crio-a51533899aab92cfcb369664a6e255b21dce681e158bc0e20b552faf24813b8a WatchSource:0}: Error finding container a51533899aab92cfcb369664a6e255b21dce681e158bc0e20b552faf24813b8a: Status 404 returned error can't find the container with id a51533899aab92cfcb369664a6e255b21dce681e158bc0e20b552faf24813b8a
Oct 30 21:14:00.982751 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:00.982721    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:01.036784 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.036750    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:14:01.037131 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.036830    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w5d85\""
Oct 30 21:14:01.214006 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.213976    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vhcn" event={"ID":"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14","Type":"ContainerStarted","Data":"6799a446a7017eda35429406297d273251a827643f0c9f7f6d5f2c405691ec57"}
Oct 30 21:14:01.214155 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.214034    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vhcn" event={"ID":"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14","Type":"ContainerStarted","Data":"fd642d2cf0fec17fd573570b7c747543289634a87bbe54d3fb539d79fe8dd3e0"}
Oct 30 21:14:01.215069 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.215046    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8fpfq" event={"ID":"830286ce-0d57-4cb7-8f13-a298b08bd319","Type":"ContainerStarted","Data":"53fcd4b90605a624857b017ef226e0d4d3eadd6aaacb51708e0abbb27390ac0e"}
Oct 30 21:14:01.216113 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.216093    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2p4x" event={"ID":"c6c54779-d916-481f-b659-55e1b3630b77","Type":"ContainerStarted","Data":"a51533899aab92cfcb369664a6e255b21dce681e158bc0e20b552faf24813b8a"}
Oct 30 21:14:01.572204 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.571164    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"]
Oct 30 21:14:01.573594 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.573567    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:01.628391 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.628072    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 21:14:01.629320 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.629286    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"]
Oct 30 21:14:01.629454 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.629431    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f9wbl\""
Oct 30 21:14:01.735170 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.735138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49b86000-eb20-4c99-9bfe-7269ea162576-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-l4g59\" (UID: \"49b86000-eb20-4c99-9bfe-7269ea162576\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:01.835955 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.835883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49b86000-eb20-4c99-9bfe-7269ea162576-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-l4g59\" (UID: \"49b86000-eb20-4c99-9bfe-7269ea162576\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:01.839218 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.839159    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49b86000-eb20-4c99-9bfe-7269ea162576-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-l4g59\" (UID: \"49b86000-eb20-4c99-9bfe-7269ea162576\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:01.886038 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:01.886010    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:02.220763 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:02.220720    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vhcn" event={"ID":"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14","Type":"ContainerStarted","Data":"d706453ca257cec0d793be3f1cabbdaa0af7decc91eaa7cd505ba831a88e07d9"}
Oct 30 21:14:02.268899 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:02.268866    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b86000_eb20_4c99_9bfe_7269ea162576.slice/crio-3a9f6bf2a7992482cac94bb210b0a2f6e7b60ef2963c40f5c63d44d699fee3de WatchSource:0}: Error finding container 3a9f6bf2a7992482cac94bb210b0a2f6e7b60ef2963c40f5c63d44d699fee3de: Status 404 returned error can't find the container with id 3a9f6bf2a7992482cac94bb210b0a2f6e7b60ef2963c40f5c63d44d699fee3de
Oct 30 21:14:02.269799 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:02.269726    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"]
Oct 30 21:14:03.225807 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:03.225771    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8fpfq" event={"ID":"830286ce-0d57-4cb7-8f13-a298b08bd319","Type":"ContainerStarted","Data":"36ebe91765174dcf25171ad0e1f7b527c6c4ca335bd6634cb1e9a4183db902de"}
Oct 30 21:14:03.226234 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:03.225831    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8fpfq" event={"ID":"830286ce-0d57-4cb7-8f13-a298b08bd319","Type":"ContainerStarted","Data":"b2475a74f823eee14b94d05f6eb516af649f7ea71d2cbbcb1288106fca672927"}
Oct 30 21:14:03.226234 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:03.225877    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:03.227328 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:03.227295    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59" event={"ID":"49b86000-eb20-4c99-9bfe-7269ea162576","Type":"ContainerStarted","Data":"3a9f6bf2a7992482cac94bb210b0a2f6e7b60ef2963c40f5c63d44d699fee3de"}
Oct 30 21:14:03.346203 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:03.346143    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-8fpfq" podStartSLOduration=2.012405358 podStartE2EDuration="4.346124889s" totalImagesPullingTime="2.333719531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:03.345828787 +0000 UTC m=+53.876192239" watchObservedRunningTime="2026-10-30 21:14:03.346124889 +0000 UTC m=+53.876488345"
Oct 30 21:14:04.233034 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.233000    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2p4x" event={"ID":"c6c54779-d916-481f-b659-55e1b3630b77","Type":"ContainerStarted","Data":"2e3b6a07a4853a43667d245dd18b730d3cdbd8b46fbcc9d2041c941fb2b5d0ac"}
Oct 30 21:14:04.235342 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.235318    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8vhcn" event={"ID":"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14","Type":"ContainerStarted","Data":"3b069aef035142520c8a6e158b77ebfeed0c01260c1fae29ea6fe8f61beba50c"}
Oct 30 21:14:04.236886 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.236858    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59" event={"ID":"49b86000-eb20-4c99-9bfe-7269ea162576","Type":"ContainerStarted","Data":"dc8a609a17dd83b04c4dc2d5afe47fc9256dab49b44983834c74f5accf5ed370"}
Oct 30 21:14:04.236975 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.236937    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:04.241890 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.241873    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:04.351796 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.351702    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q2p4x" podStartSLOduration=1.8729722500000001 podStartE2EDuration="5.351689411s" totalImagesPullingTime="3.478717161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:04.351312845 +0000 UTC m=+54.881676301" watchObservedRunningTime="2026-10-30 21:14:04.351689411 +0000 UTC m=+54.882052863"
Oct 30 21:14:04.472259 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.472194    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8vhcn" podStartSLOduration=2.159739444 podStartE2EDuration="5.472180625s" totalImagesPullingTime="3.312441181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:13:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:04.471828586 +0000 UTC m=+55.002192047" watchObservedRunningTime="2026-10-30 21:14:04.472180625 +0000 UTC m=+55.002544079"
Oct 30 21:14:04.590176 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:04.590123    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59" podStartSLOduration=1.8649750250000001 podStartE2EDuration="3.590109042s" totalImagesPullingTime="1.725134017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:04.589523073 +0000 UTC m=+55.119886527" watchObservedRunningTime="2026-10-30 21:14:04.590109042 +0000 UTC m=+55.120472497"
Oct 30 21:14:06.149951 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.149922    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"]
Oct 30 21:14:06.151848 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.151832    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.255924 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.255888    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:14:06.255924 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.255894    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 21:14:06.256140 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.255938    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:14:06.256140 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.256054    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 21:14:06.256812 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.256794    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"]
Oct 30 21:14:06.258879 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.258864    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xx59j\""
Oct 30 21:14:06.268297 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.268259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n82bm\" (UniqueName: \"kubernetes.io/projected/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-kube-api-access-n82bm\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.268445 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.268327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.268445 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.268402    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.268445 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.268424    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.306769 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.306747    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:14:06.369216 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.369185    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n82bm\" (UniqueName: \"kubernetes.io/projected/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-kube-api-access-n82bm\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.369310 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.369237    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.369310 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.369259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.369310 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.369276    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.370022 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.369998    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.371670 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.371649    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.371791 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.371774    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.431609 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.431534    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n82bm\" (UniqueName: \"kubernetes.io/projected/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-kube-api-access-n82bm\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.462404 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.462370    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:06.836282 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:06.836250    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"]
Oct 30 21:14:06.853980 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:06.853955    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1ba61a_eb5d_4b06_8690_b564a17d47fd.slice/crio-07dee1aaf5e446463af3845d15f7f105c9a5c164cc2f61808360e276361f7d74 WatchSource:0}: Error finding container 07dee1aaf5e446463af3845d15f7f105c9a5c164cc2f61808360e276361f7d74: Status 404 returned error can't find the container with id 07dee1aaf5e446463af3845d15f7f105c9a5c164cc2f61808360e276361f7d74
Oct 30 21:14:07.174026 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:07.173945    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:07.246336 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:07.246301    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv" event={"ID":"ff1ba61a-eb5d-4b06-8690-b564a17d47fd","Type":"ContainerStarted","Data":"07dee1aaf5e446463af3845d15f7f105c9a5c164cc2f61808360e276361f7d74"}
Oct 30 21:14:09.253705 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:09.253668    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv" event={"ID":"ff1ba61a-eb5d-4b06-8690-b564a17d47fd","Type":"ContainerStarted","Data":"69ea3ed348582d900bb5e565c482fc98c8ac310c3d3da412b7b8eb453f67415e"}
Oct 30 21:14:09.254186 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:09.253729    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv" event={"ID":"ff1ba61a-eb5d-4b06-8690-b564a17d47fd","Type":"ContainerStarted","Data":"7a14ee6e95a00740d9212fa10559a5b6cd9d534c6a46b1972794afe18b4d4b82"}
Oct 30 21:14:09.375153 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:09.375090    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv" podStartSLOduration=1.574050323 podStartE2EDuration="3.375075312s" totalImagesPullingTime="1.801024989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:09.37449324 +0000 UTC m=+59.904856696" watchObservedRunningTime="2026-10-30 21:14:09.375075312 +0000 UTC m=+59.905438763"
Oct 30 21:14:12.650418 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.650382    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"]
Oct 30 21:14:12.652839 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.652823    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.713654 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.713609    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:14:12.713654 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.713611    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-p88cx\""
Oct 30 21:14:12.713896 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.713715    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:14:12.716022 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.715988    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"]
Oct 30 21:14:12.790531 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.790501    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t47rj"]
Oct 30 21:14:12.792530 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.792513    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"]
Oct 30 21:14:12.792662 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.792645    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.794543 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.794522    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:12.813891 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.813870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbff937-2134-4112-8ff4-3c50ed16ffe2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.813990 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.813923    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.814084 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.814066    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.814119 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.814099    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc49h\" (UniqueName: \"kubernetes.io/projected/edbff937-2134-4112-8ff4-3c50ed16ffe2-kube-api-access-rc49h\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.849963 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.849935    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:14:12.850059 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.849990    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:14:12.869867 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.869841    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"]
Oct 30 21:14:12.914385 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914299    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a99b0768-fe06-4d64-95cb-64f5334596f8-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:12.914385 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914331    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-textfile\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.914385 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqndf\" (UniqueName: \"kubernetes.io/projected/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-kube-api-access-rqndf\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.914599 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914462    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-root\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.914599 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914495    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:14:12.914599 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914508    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:12.914599 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.914753 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914600    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:14:12.914753 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9qjj\" (UniqueName: \"kubernetes.io/projected/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-api-access-l9qjj\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:12.914753 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914646    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-metrics-client-ca\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.914753 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914747    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.914927 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:12.914927 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc49h\" (UniqueName: \"kubernetes.io/projected/edbff937-2134-4112-8ff4-3c50ed16ffe2-kube-api-access-rc49h\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.914927 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbff937-2134-4112-8ff4-3c50ed16ffe2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.914927 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914865    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xhltr\""
Oct 30 21:14:12.915109 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.914979    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:14:12.915109 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:12.915109 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.915109 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915059    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w5xnx\""
Oct 30 21:14:12.915109 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915059    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:14:12.915365 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-sys\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.915365 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915142    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-wtmp\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.915365 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.915365 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:12.915365 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-tls\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:12.915691 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.915671    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbff937-2134-4112-8ff4-3c50ed16ffe2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.917593 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.917575    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.917671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.917657    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:12.976608 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:12.976578    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc49h\" (UniqueName: \"kubernetes.io/projected/edbff937-2134-4112-8ff4-3c50ed16ffe2-kube-api-access-rc49h\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016211    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9qjj\" (UniqueName: \"kubernetes.io/projected/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-api-access-l9qjj\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-metrics-client-ca\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016433    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-sys\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016570    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-wtmp\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-tls\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a99b0768-fe06-4d64-95cb-64f5334596f8-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-textfile\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016750    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqndf\" (UniqueName: \"kubernetes.io/projected/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-kube-api-access-rqndf\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-root\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.017521 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.016812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.019668 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.019634    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-root\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.019812 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.019691    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-sys\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.019812 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.019747    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-wtmp\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.019928 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.019844    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-textfile\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.020361 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.020242    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.020478 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.020332    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-metrics-client-ca\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.020478 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.020242    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.020478 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.020411    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.020657 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.020630    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a99b0768-fe06-4d64-95cb-64f5334596f8-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.021093 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.021073    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.022173 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.022155    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.022577 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.022553    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-tls\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.022656 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.022568    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.086288 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.086255    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqndf\" (UniqueName: \"kubernetes.io/projected/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-kube-api-access-rqndf\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.089288 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.089258    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9qjj\" (UniqueName: \"kubernetes.io/projected/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-api-access-l9qjj\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.106132 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.106110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:13.110470 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.110454    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:13.116889 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:13.116862    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807ebbbd_72a2_47c3_b50a_2b044f71a8ff.slice/crio-d542cf155686da3099789ecb97a85ed998892431c28cd1c34e0716b9f0820fbe WatchSource:0}: Error finding container d542cf155686da3099789ecb97a85ed998892431c28cd1c34e0716b9f0820fbe: Status 404 returned error can't find the container with id d542cf155686da3099789ecb97a85ed998892431c28cd1c34e0716b9f0820fbe
Oct 30 21:14:13.239408 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.239376    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:13.264277 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.264249    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:13.265475 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.265447    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t47rj" event={"ID":"807ebbbd-72a2-47c3-b50a-2b044f71a8ff","Type":"ContainerStarted","Data":"d542cf155686da3099789ecb97a85ed998892431c28cd1c34e0716b9f0820fbe"}
Oct 30 21:14:13.525873 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:13.525818    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99b0768_fe06_4d64_95cb_64f5334596f8.slice/crio-099f62e01b20b261bc3dc27df9cef129812d4a1427f8343ddfab1aa32607e654 WatchSource:0}: Error finding container 099f62e01b20b261bc3dc27df9cef129812d4a1427f8343ddfab1aa32607e654: Status 404 returned error can't find the container with id 099f62e01b20b261bc3dc27df9cef129812d4a1427f8343ddfab1aa32607e654
Oct 30 21:14:13.528022 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.527997    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"]
Oct 30 21:14:13.638173 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:13.638138    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbff937_2134_4112_8ff4_3c50ed16ffe2.slice/crio-e59bd031a441c1b7049a66d6e0392e6b82a3eac0e1965d2b1a18236bb71f2f70 WatchSource:0}: Error finding container e59bd031a441c1b7049a66d6e0392e6b82a3eac0e1965d2b1a18236bb71f2f70: Status 404 returned error can't find the container with id e59bd031a441c1b7049a66d6e0392e6b82a3eac0e1965d2b1a18236bb71f2f70
Oct 30 21:14:13.639419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:13.639400    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"]
Oct 30 21:14:14.271041 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:14.271004    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh" event={"ID":"edbff937-2134-4112-8ff4-3c50ed16ffe2","Type":"ContainerStarted","Data":"a8a434725f8db3e460eb3dcb3083151bb92ac6652e133492b27b92cfa436f25c"}
Oct 30 21:14:14.271509 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:14.271078    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh" event={"ID":"edbff937-2134-4112-8ff4-3c50ed16ffe2","Type":"ContainerStarted","Data":"a582db487a0bbf307b511383294df915bb474811c4316fd19857342f76e81df7"}
Oct 30 21:14:14.271509 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:14.271108    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh" event={"ID":"edbff937-2134-4112-8ff4-3c50ed16ffe2","Type":"ContainerStarted","Data":"e59bd031a441c1b7049a66d6e0392e6b82a3eac0e1965d2b1a18236bb71f2f70"}
Oct 30 21:14:14.272475 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:14.272448    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk" event={"ID":"a99b0768-fe06-4d64-95cb-64f5334596f8","Type":"ContainerStarted","Data":"099f62e01b20b261bc3dc27df9cef129812d4a1427f8343ddfab1aa32607e654"}
Oct 30 21:14:15.110769 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.110733    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:14:15.115004 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.114977    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.226553 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.226522    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:14:15.226723 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.226522    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:14:15.226723 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.226594    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:14:15.226723 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.226648    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:14:15.226723 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.226660    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:14:15.226723 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.226711    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h6tkb\""
Oct 30 21:14:15.226955 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.226725    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:14:15.228094 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.228068    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:14:15.236276 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236412 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236489 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236489 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236461    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236580 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236612 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236590    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236647 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236723 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxqrw\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236800 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236783    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236856 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236821    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236856 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236850    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.236948 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.237000 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.236976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.277328 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.277297    2574 generic.go:356] "Generic (PLEG): container finished" podID="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" containerID="843ac632d5ef41bd8c1425988fa256358df9800b8b2d79b3910737f54a05d44a" exitCode=0
Oct 30 21:14:15.277724 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.277360    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t47rj" event={"ID":"807ebbbd-72a2-47c3-b50a-2b044f71a8ff","Type":"ContainerDied","Data":"843ac632d5ef41bd8c1425988fa256358df9800b8b2d79b3910737f54a05d44a"}
Oct 30 21:14:15.306719 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.306692    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:14:15.306826 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.306739    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:14:15.306826 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.306740    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:14:15.337761 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.337738    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.337862 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.337780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.337862 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.337804    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.337974 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.337951    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338024 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.337990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338024 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqrw\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338122 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338043    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338122 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338208 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338137    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338208 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338208 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338373 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.338373 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338278    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.339409 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.338655    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.339409 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.339367    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.339998 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.339955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.341322 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.341122    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.342948 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.341560    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.342948 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.341819    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.342948 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.342226    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.342948 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.342293    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.342948 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.342804    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.343297 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.343273    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.344100 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.344078    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.344272 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.344196    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.359992 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.359970    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5c84f565c4-h97ps"]
Oct 30 21:14:15.363419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.363377    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.402314 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.402290    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqrw\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.442230 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.442203    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:15.442947 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.442923    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c84f565c4-h97ps"]
Oct 30 21:14:15.446706 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.444086    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-47rmpbtts3l4q\""
Oct 30 21:14:15.446706 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.444215    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:14:15.446706 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.444252    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:14:15.446706 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.444328    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-m55dr\""
Oct 30 21:14:15.446706 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.444372    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:14:15.446706 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.444424    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:14:15.446706 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.444486    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:14:15.540654 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540627    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540713 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540690    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2hj\" (UniqueName: \"kubernetes.io/projected/0bacc5cd-6d42-48f6-8807-5117835ca615-kube-api-access-kl2hj\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540759 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540759 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540730    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bacc5cd-6d42-48f6-8807-5117835ca615-metrics-client-ca\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540759 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-grpc-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540844 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540844 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bacc5cd-6d42-48f6-8807-5117835ca615-tmp\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540844 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.540929 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.540848    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641682 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2hj\" (UniqueName: \"kubernetes.io/projected/0bacc5cd-6d42-48f6-8807-5117835ca615-kube-api-access-kl2hj\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641682 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641682 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bacc5cd-6d42-48f6-8807-5117835ca615-metrics-client-ca\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641986 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641693    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-grpc-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641986 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641986 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641759    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bacc5cd-6d42-48f6-8807-5117835ca615-tmp\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641986 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641781    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641986 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.641986 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.641869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.642270 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.642207    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bacc5cd-6d42-48f6-8807-5117835ca615-tmp\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.642457 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.642412    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bacc5cd-6d42-48f6-8807-5117835ca615-metrics-client-ca\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.644327 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.644299    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.644464 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.644439    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.644862 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.644835    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.644935 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.644867    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-grpc-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.644935 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.644904    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.645007 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.644939    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.704600 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.704573    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2hj\" (UniqueName: \"kubernetes.io/projected/0bacc5cd-6d42-48f6-8807-5117835ca615-kube-api-access-kl2hj\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:15.742434 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.742402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:15.742582 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.742450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:15.795622 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.795593    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:14:15.795622 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.795617    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:14:15.804966 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.804937    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:15.804966 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.804952    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:15.882058 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:15.882031    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9adc37_ec78_4112_93c6_e8e02e34bfd2.slice/crio-44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a WatchSource:0}: Error finding container 44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a: Status 404 returned error can't find the container with id 44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a
Oct 30 21:14:15.882610 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.882584    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:14:15.944216 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.944137    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:15.948419 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.948400    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q4s27\""
Oct 30 21:14:15.956298 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.956280    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:16.000055 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:15.999394    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:16.004932 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.004913    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:14:16.045815 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.045759    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w5d85\""
Oct 30 21:14:16.054614 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.054412    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:16.068903 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.068876    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:14:16.078264 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.078235    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:16.252071 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.252042    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5fn5j\""
Oct 30 21:14:16.260272 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.260244    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:16.281981 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.281941    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerStarted","Data":"44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a"}
Oct 30 21:14:16.284563 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.284536    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk" event={"ID":"a99b0768-fe06-4d64-95cb-64f5334596f8","Type":"ContainerStarted","Data":"9a9789dbebf26b9ce3e8e41f3708db39caf8aa2a9373a2247379840d56d6ab2f"}
Oct 30 21:14:16.284873 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.284593    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk" event={"ID":"a99b0768-fe06-4d64-95cb-64f5334596f8","Type":"ContainerStarted","Data":"1e4d86789e922f8ac2208ab1183367d182015b9c3cf88bc2dcfd8920d4654715"}
Oct 30 21:14:16.284873 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.284621    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk" event={"ID":"a99b0768-fe06-4d64-95cb-64f5334596f8","Type":"ContainerStarted","Data":"28f1294b85e48beb20059a9ac6bc9e1f24c2e5c246f4ecda140222d24c178212"}
Oct 30 21:14:16.286992 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.286965    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t47rj" event={"ID":"807ebbbd-72a2-47c3-b50a-2b044f71a8ff","Type":"ContainerStarted","Data":"ca031d3191fca99d787bac626e27a99466641733c2d79f6ed214ebe403c15fed"}
Oct 30 21:14:16.287087 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.287015    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t47rj" event={"ID":"807ebbbd-72a2-47c3-b50a-2b044f71a8ff","Type":"ContainerStarted","Data":"443660041da64f0f0114c61328b38687649774336d9a9a83a463cc7387af74a5"}
Oct 30 21:14:16.289388 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.289335    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh" event={"ID":"edbff937-2134-4112-8ff4-3c50ed16ffe2","Type":"ContainerStarted","Data":"82ff45529f3257c4019e0b11ecbab80a0197783e1aa6386d513d609815baa96b"}
Oct 30 21:14:16.334263 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.334238    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-26g9z"]
Oct 30 21:14:16.337136 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:16.336882    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e048855_3db9_47b4_a1dc_29ee474f05e0.slice/crio-f945fdec6e7d13e452db60d6f5c4a70841659083f0f1b5a5045c8154ba0b04a7 WatchSource:0}: Error finding container f945fdec6e7d13e452db60d6f5c4a70841659083f0f1b5a5045c8154ba0b04a7: Status 404 returned error can't find the container with id f945fdec6e7d13e452db60d6f5c4a70841659083f0f1b5a5045c8154ba0b04a7
Oct 30 21:14:16.391823 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.391792    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-npz4c"]
Oct 30 21:14:16.396232 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.396204    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:16.408006 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.407958    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk" podStartSLOduration=2.544429955 podStartE2EDuration="4.407945287s" totalImagesPullingTime="1.863515332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:16.407280063 +0000 UTC m=+66.937643517" watchObservedRunningTime="2026-10-30 21:14:16.407945287 +0000 UTC m=+66.938308742"
Oct 30 21:14:16.439626 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:16.439588    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bacc5cd_6d42_48f6_8807_5117835ca615.slice/crio-c3f1549025938864c1d9ce58ad78a46679d596daf883de0e1739e34deb35f66a WatchSource:0}: Error finding container c3f1549025938864c1d9ce58ad78a46679d596daf883de0e1739e34deb35f66a: Status 404 returned error can't find the container with id c3f1549025938864c1d9ce58ad78a46679d596daf883de0e1739e34deb35f66a
Oct 30 21:14:16.440561 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.440536    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c84f565c4-h97ps"]
Oct 30 21:14:16.461494 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.461469    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:14:16.461601 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.461500    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:14:16.470625 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.470599    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-5mkrp\""
Oct 30 21:14:16.483740 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.483712    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-npz4c"]
Oct 30 21:14:16.504027 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:16.503961    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b98cf0_3a8a_46b3_ac91_f18817011ee8.slice/crio-c8aed7a3fa9d29a172172477906bbd396d25e2c7fe16ce3c113f2350c6dbc362 WatchSource:0}: Error finding container c8aed7a3fa9d29a172172477906bbd396d25e2c7fe16ce3c113f2350c6dbc362: Status 404 returned error can't find the container with id c8aed7a3fa9d29a172172477906bbd396d25e2c7fe16ce3c113f2350c6dbc362
Oct 30 21:14:16.505466 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.505445    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxrjh"]
Oct 30 21:14:16.537917 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.537871    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh" podStartSLOduration=2.944761229 podStartE2EDuration="4.537859166s" totalImagesPullingTime="1.593097937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:16.533427794 +0000 UTC m=+67.063791274" watchObservedRunningTime="2026-10-30 21:14:16.537859166 +0000 UTC m=+67.068222621"
Oct 30 21:14:16.549624 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.549598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9bhq\" (UniqueName: \"kubernetes.io/projected/4ecf5604-81f2-4892-85a4-b2ac4dacf993-kube-api-access-k9bhq\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:16.549746 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.549640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecf5604-81f2-4892-85a4-b2ac4dacf993-tmp\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:16.640968 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:16.640939    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e2f746_f082_4371_b1c0_375a66cf81da.slice/crio-f9d1f190488187e7cefad540ed43247a7ef98cca6a821bbb247a59e273609a5d WatchSource:0}: Error finding container f9d1f190488187e7cefad540ed43247a7ef98cca6a821bbb247a59e273609a5d: Status 404 returned error can't find the container with id f9d1f190488187e7cefad540ed43247a7ef98cca6a821bbb247a59e273609a5d
Oct 30 21:14:16.641676 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.641636    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jg5k9"]
Oct 30 21:14:16.650161 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.650127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9bhq\" (UniqueName: \"kubernetes.io/projected/4ecf5604-81f2-4892-85a4-b2ac4dacf993-kube-api-access-k9bhq\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:16.650248 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.650166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecf5604-81f2-4892-85a4-b2ac4dacf993-tmp\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:16.650578 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.650562    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecf5604-81f2-4892-85a4-b2ac4dacf993-tmp\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:16.661635 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.661597    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t47rj" podStartSLOduration=3.088680876 podStartE2EDuration="4.661583917s" totalImagesPullingTime="1.492819936s" totalInitContainerRuntime="80.083105ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:16.660818727 +0000 UTC m=+67.191182184" watchObservedRunningTime="2026-10-30 21:14:16.661583917 +0000 UTC m=+67.191947371"
Oct 30 21:14:16.718829 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:16.718801    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9bhq\" (UniqueName: \"kubernetes.io/projected/4ecf5604-81f2-4892-85a4-b2ac4dacf993-kube-api-access-k9bhq\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:17.007881 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.007847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:17.295099 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.295014    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxrjh" event={"ID":"73b98cf0-3a8a-46b3-ac91-f18817011ee8","Type":"ContainerStarted","Data":"c8aed7a3fa9d29a172172477906bbd396d25e2c7fe16ce3c113f2350c6dbc362"}
Oct 30 21:14:17.296336 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.296306    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jg5k9" event={"ID":"45e2f746-f082-4371-b1c0-375a66cf81da","Type":"ContainerStarted","Data":"f9d1f190488187e7cefad540ed43247a7ef98cca6a821bbb247a59e273609a5d"}
Oct 30 21:14:17.299098 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.299069    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" event={"ID":"0bacc5cd-6d42-48f6-8807-5117835ca615","Type":"ContainerStarted","Data":"c3f1549025938864c1d9ce58ad78a46679d596daf883de0e1739e34deb35f66a"}
Oct 30 21:14:17.301167 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.301131    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-26g9z" event={"ID":"2e048855-3db9-47b4-a1dc-29ee474f05e0","Type":"ContainerStarted","Data":"f945fdec6e7d13e452db60d6f5c4a70841659083f0f1b5a5045c8154ba0b04a7"}
Oct 30 21:14:17.390716 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:17.390688    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecf5604_81f2_4892_85a4_b2ac4dacf993.slice/crio-d89ce32aa46061b2561ae609b3da7f02d8b8d42efa56d6afa0dbafa82b91c85a WatchSource:0}: Error finding container d89ce32aa46061b2561ae609b3da7f02d8b8d42efa56d6afa0dbafa82b91c85a: Status 404 returned error can't find the container with id d89ce32aa46061b2561ae609b3da7f02d8b8d42efa56d6afa0dbafa82b91c85a
Oct 30 21:14:17.394060 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.392877    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"]
Oct 30 21:14:17.396888 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.396867    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-npz4c"]
Oct 30 21:14:17.396982 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.396945    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:17.451582 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.451546    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"]
Oct 30 21:14:17.516802 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.516681    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:14:17.516802 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.516772    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-j7gnd\""
Oct 30 21:14:17.558083 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.557910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:17.558083 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.557980    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-667fc\" (UniqueName: \"kubernetes.io/projected/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-kube-api-access-667fc\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:17.659623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.659589    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-667fc\" (UniqueName: \"kubernetes.io/projected/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-kube-api-access-667fc\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:17.659770 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.659674    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:17.668886 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.668811    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:17.739941 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:17.739880    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-667fc\" (UniqueName: \"kubernetes.io/projected/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-kube-api-access-667fc\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:18.010988 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:18.010952    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:18.308853 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:18.308816    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-npz4c" event={"ID":"4ecf5604-81f2-4892-85a4-b2ac4dacf993","Type":"ContainerStarted","Data":"d89ce32aa46061b2561ae609b3da7f02d8b8d42efa56d6afa0dbafa82b91c85a"}
Oct 30 21:14:18.313628 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:18.313598    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016" exitCode=0
Oct 30 21:14:18.313762 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:18.313660    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"}
Oct 30 21:14:18.390374 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:18.389727    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"]
Oct 30 21:14:18.391729 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:18.391700    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b75f40_bfd6_4ed1_88e8_0c56ee7d41f7.slice/crio-d8635e6f3ade4b2e186e11dfff94108a77e85a4d877ae920b4ce2e818395340a WatchSource:0}: Error finding container d8635e6f3ade4b2e186e11dfff94108a77e85a4d877ae920b4ce2e818395340a: Status 404 returned error can't find the container with id d8635e6f3ade4b2e186e11dfff94108a77e85a4d877ae920b4ce2e818395340a
Oct 30 21:14:19.324269 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:19.323394    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxrjh" event={"ID":"73b98cf0-3a8a-46b3-ac91-f18817011ee8","Type":"ContainerStarted","Data":"02f5e812822f9072f6bda9dcf770672f08758bf2d328f0b4a528b29698ee2517"}
Oct 30 21:14:19.324269 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:19.323511    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxrjh" event={"ID":"73b98cf0-3a8a-46b3-ac91-f18817011ee8","Type":"ContainerStarted","Data":"d150e85313839a6546001947da57480044f10d9f446234b858415fff5704d032"}
Oct 30 21:14:19.325817 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:19.325745    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5" event={"ID":"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7","Type":"ContainerStarted","Data":"d8635e6f3ade4b2e186e11dfff94108a77e85a4d877ae920b4ce2e818395340a"}
Oct 30 21:14:19.444304 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:19.444251    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fxrjh" podStartSLOduration=67.476493254 podStartE2EDuration="1m9.44423606s" totalImagesPullingTime="1.967742806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:19.443142812 +0000 UTC m=+69.973506290" watchObservedRunningTime="2026-10-30 21:14:19.44423606 +0000 UTC m=+69.974599514"
Oct 30 21:14:21.347623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:21.347394    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" event={"ID":"0bacc5cd-6d42-48f6-8807-5117835ca615","Type":"ContainerStarted","Data":"6e6ecfdecc5bbb275106b05586703c31c66375f980282ee3c919a04428647e1b"}
Oct 30 21:14:21.347623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:21.347532    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" event={"ID":"0bacc5cd-6d42-48f6-8807-5117835ca615","Type":"ContainerStarted","Data":"9633cebac15c1efff8c5931a6ad5f3eaf37e07bd18b01b6044e9122f174cb394"}
Oct 30 21:14:21.347623 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:21.347574    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" event={"ID":"0bacc5cd-6d42-48f6-8807-5117835ca615","Type":"ContainerStarted","Data":"0c06d47a4273c3f2b04ce4e4d4e517a2aca05c4cad0079b67cc83151c2ee6d6e"}
Oct 30 21:14:22.292427 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.292044    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:14:22.300638 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.299649    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307405    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307455    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307482    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307540    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307561    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307612    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307634    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zrlw\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.307694 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.307656    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309251 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309251 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309420 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309404    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309473 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309453    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309522 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309522 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309514    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309611 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309562    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309611 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309595    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309707 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.309707 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.309663    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.354253 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.354228    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:14:22.354640 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.354321    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:14:22.354640 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.354409    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-68m835m929agv\""
Oct 30 21:14:22.354640 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.354466    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k9bmv\""
Oct 30 21:14:22.354640 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.354591    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:14:22.354808 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.354766    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:14:22.354844 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.354813    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:14:22.360202 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.360171    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" event={"ID":"0bacc5cd-6d42-48f6-8807-5117835ca615","Type":"ContainerStarted","Data":"d32a4b232026cd6a728fcf4516c85fde6592c7a03896361a78230e64a0be20d9"}
Oct 30 21:14:22.366048 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.363793    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5" event={"ID":"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7","Type":"ContainerStarted","Data":"541e7ef5ae6c9643872b1739c27f610bf8a4d8968875e09713bd7a2ae1554e21"}
Oct 30 21:14:22.366048 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.365162    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:22.371816 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.371797    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:22.375609 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.375585    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerStarted","Data":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"}
Oct 30 21:14:22.375991 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.375650    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerStarted","Data":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"}
Oct 30 21:14:22.375991 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.375693    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerStarted","Data":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"}
Oct 30 21:14:22.375991 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.375731    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerStarted","Data":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"}
Oct 30 21:14:22.375991 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.375765    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerStarted","Data":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"}
Oct 30 21:14:22.406616 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.406590    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:14:22.415049 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415145 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415145 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415121    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415255 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415255 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415255 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415239    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415436 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415436 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415436 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415324    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415436 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415436 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415464    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415516    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zrlw\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415671 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415981 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415674    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.415981 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.415715    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.418425 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.416634    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.418425 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.417187    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.418425 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.417581    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.418425 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.418132    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.419100 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.419080    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.419539 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.419518    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.421506 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.421468    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.422023 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.421970    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.423311 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.423268    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.454865 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.454835    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:14:22.455002 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.454864    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:14:22.455002 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.454929    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:14:22.455002 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.454938    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:14:22.455127 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.455105    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:14:22.455196 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.455177    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:14:22.455259 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.455180    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:14:22.455259 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.455240    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:14:22.457375 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.456671    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.457712 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.457468    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.459508 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.459016    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.460713 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.460579    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.460713 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.460596    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.461456 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.461236    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.463698 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.463651    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.469210 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.469161    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.521920 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.518467    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:14:22.521920 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.518661    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:14:22.521920 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.521822    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zrlw\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.521920 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.521832    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.527409 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.527112    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:22.638037 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:22.638005    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:23.086677 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.086625    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5" podStartSLOduration=3.01414483 podStartE2EDuration="6.086604147s" totalImagesPullingTime="3.072459317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:22.674491277 +0000 UTC m=+73.204854733" watchObservedRunningTime="2026-10-30 21:14:23.086604147 +0000 UTC m=+73.616967599"
Oct 30 21:14:23.087085 ip-10-0-142-25 kubenswrapper[2574]: W1030 21:14:23.087058    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b3c88c_4121_4a5b_acb2_8d7e78a71243.slice/crio-64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6 WatchSource:0}: Error finding container 64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6: Status 404 returned error can't find the container with id 64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6
Oct 30 21:14:23.088622 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.088490    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:14:23.384436 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.383635    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jg5k9" event={"ID":"45e2f746-f082-4371-b1c0-375a66cf81da","Type":"ContainerStarted","Data":"403526e9b4330fc7d18bfad8ea526b14e13dc52f5338e2c6e96182e1aa206d45"}
Oct 30 21:14:23.385470 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.384975    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:23.390141 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.389164    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" event={"ID":"0bacc5cd-6d42-48f6-8807-5117835ca615","Type":"ContainerStarted","Data":"689994f7baacaabf5f42c46f1d74b1ca45a4caded9e78a32c95963e8c0dded9b"}
Oct 30 21:14:23.390141 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.389236    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" event={"ID":"0bacc5cd-6d42-48f6-8807-5117835ca615","Type":"ContainerStarted","Data":"023ea0530b8495100b5586c2c51fe5a3dc73deea81f0473affb32b0c75df99f0"}
Oct 30 21:14:23.390141 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.390072    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:23.394959 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.394892    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerStarted","Data":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"}
Oct 30 21:14:23.397689 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.397374    2574 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244" exitCode=0
Oct 30 21:14:23.397689 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.397418    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"}
Oct 30 21:14:23.397689 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.397491    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerStarted","Data":"64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6"}
Oct 30 21:14:23.504022 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.503957    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jg5k9" podStartSLOduration=67.397708017 podStartE2EDuration="1m13.50393722s" totalImagesPullingTime="6.106229203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:23.502526551 +0000 UTC m=+74.032890007" watchObservedRunningTime="2026-10-30 21:14:23.50393722 +0000 UTC m=+74.034300677"
Oct 30 21:14:23.775399 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.775304    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.819784086 podStartE2EDuration="8.77528691s" totalImagesPullingTime="5.940093377s" totalInitContainerRuntime="15.409447ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:23.773104429 +0000 UTC m=+74.303467885" watchObservedRunningTime="2026-10-30 21:14:23.77528691 +0000 UTC m=+74.305650365"
Oct 30 21:14:23.904125 ip-10-0-142-25 kubenswrapper[2574]: I1030 21:14:23.904069    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps" podStartSLOduration=3.385413405 podStartE2EDuration="8.904054468s" totalImagesPullingTime="5.518641063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:23.903463713 +0000 UTC m=+74.433827168" watchObservedRunningTime="2026-10-30 21:14:23.904054468 +0000 UTC m=+74.434417922"
Oct 30 21:14:24.659590 ip-10-0-142-25 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:14:24.668145 ip-10-0-142-25 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:14:24.668422 ip-10-0-142-25 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:14:24.668482 ip-10-0-142-25 systemd[1]: kubelet.service: Consumed 4.303s CPU time.
Oct 30 21:14:24.679930 ip-10-0-142-25 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:14:24.744621 ip-10-0-142-25 kubenswrapper[5817]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:14:24.744621 ip-10-0-142-25 kubenswrapper[5817]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:14:24.744621 ip-10-0-142-25 kubenswrapper[5817]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:14:24.744621 ip-10-0-142-25 kubenswrapper[5817]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:14:24.746773 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746688    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.746773 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746771    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.746773 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746776    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746780    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746783    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746786    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746789    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746792    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746794    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746797    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746800    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746802    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746805    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746808    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746811    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746824    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746829    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746833    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746836    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746838    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746841    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746844    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.746891 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746846    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746849    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746852    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746854    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746857    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746859    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746862    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746866    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746869    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746872    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746876    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746879    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746881    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746884    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746887    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746890    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746892    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746895    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746898    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.747461 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746902    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746905    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746908    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746910    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746912    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746915    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746918    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746921    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746924    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746926    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746929    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746932    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746934    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746937    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746939    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746942    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746944    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746947    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746949    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.747967 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746952    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746955    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746958    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746960    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746963    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746966    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746968    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746971    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746974    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746976    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746979    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746981    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746984    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746987    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746989    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746992    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746994    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746997    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.746999    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.748479 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747002    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747004    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747008    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747015    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747017    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747020    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747023    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747025    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747028    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747031    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747033    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747036    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747038    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747041    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747043    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747046    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747048    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747051    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747054    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747056    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.748988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747059    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747062    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747065    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747068    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747071    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747073    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.747076    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748144    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748155    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748160    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748164    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748168    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748172    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748177    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748181    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748189    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748193    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748200    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748206    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.749536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748211    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748215    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748218    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748223    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748227    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748231    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748235    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748239    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748243    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748247    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748251    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748256    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748260    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748264    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748269    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748274    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748277    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748281    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748285    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.750134 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748290    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748294    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748298    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748302    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748306    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748310    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748314    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748319    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748322    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748328    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748333    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748337    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748341    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748363    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748368    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748372    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748376    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748380    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748385    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748389    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.750984 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748393    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748397    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748401    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748405    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748410    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748414    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748418    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748423    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748427    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748432    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748437    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748441    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748445    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748449    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748454    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748458    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748462    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748466    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748470    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748474    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.751713 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748478    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748484    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748489    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748494    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748499    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748503    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748508    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748512    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748517    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748521    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748525    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748531    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748537    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748541    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748545    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748550    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748554    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748558    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748563    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.752289 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748567    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748571    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748575    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748579    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748583    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748587    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748591    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748595    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748599    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748603    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748607    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748612    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748616    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748620    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748625    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.748629    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749201    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749209    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749212    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749216    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.752815 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749218    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749221    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749224    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749227    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749230    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749232    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749237    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749241    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749244    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749248    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749251    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749253    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749256    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749259    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749261    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749264    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749267    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749269    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749272    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.753386 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749274    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749277    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749280    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749282    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749285    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749287    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749290    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749295    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749298    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749300    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749303    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749306    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749309    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749311    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749314    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749317    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749319    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749322    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749325    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.753890 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749328    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749330    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749333    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749336    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749339    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749357    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749362    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749366    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749371    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749373    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749376    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749379    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749382    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749385    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749388    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749390    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749393    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749395    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749398    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749401    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.754652 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749405    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749408    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749411    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749413    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749417    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749420    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749422    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749425    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749428    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749430    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749433    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749435    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749438    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749441    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749444    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749448    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749452    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749455    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749459    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.755163 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749463    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749467    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749471    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749475    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749479    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749483    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749487    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749496    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749500    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749503    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749507    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749511    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749515    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749522    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749526    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749530    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749534    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749538    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749543    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749547    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.755666 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749551    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749555    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749560    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749566    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.749574    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749699    5817 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749719    5817 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749729    5817 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749736    5817 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749742    5817 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749747    5817 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749753    5817 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749759    5817 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749764    5817 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749769    5817 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749775    5817 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749780    5817 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749785    5817 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749789    5817 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749794    5817 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749799    5817 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749804    5817 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749809    5817 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:14:24.756219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749815    5817 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749820    5817 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749826    5817 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749831    5817 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749836    5817 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749842    5817 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749846    5817 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749851    5817 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749856    5817 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749861    5817 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749866    5817 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749871    5817 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749878    5817 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749882    5817 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749890    5817 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749895    5817 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749900    5817 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749905    5817 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749909    5817 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749914    5817 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749922    5817 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749927    5817 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749932    5817 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749937    5817 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:14:24.756803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749942    5817 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749949    5817 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749953    5817 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749958    5817 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749963    5817 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749967    5817 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749972    5817 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749977    5817 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749981    5817 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749984    5817 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749987    5817 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749990    5817 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749994    5817 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.749998    5817 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750002    5817 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750005    5817 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750008    5817 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750012    5817 flags.go:64] FLAG: --help="false"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750015    5817 flags.go:64] FLAG: --hostname-override="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750018    5817 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750024    5817 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750027    5817 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750030    5817 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:14:24.757694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750034    5817 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750037    5817 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750040    5817 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750042    5817 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750045    5817 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750048    5817 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750052    5817 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750054    5817 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750058    5817 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750060    5817 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750064    5817 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750067    5817 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750070    5817 flags.go:64] FLAG: --lock-file=""
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750073    5817 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750076    5817 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750079    5817 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750085    5817 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750088    5817 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750091    5817 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750094    5817 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750097    5817 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750101    5817 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750104    5817 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750107    5817 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750111    5817 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:14:24.758385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750114    5817 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750118    5817 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750122    5817 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750125    5817 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750129    5817 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750133    5817 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750136    5817 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750139    5817 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750142    5817 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750149    5817 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750153    5817 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750156    5817 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750159    5817 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750164    5817 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750168    5817 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750173    5817 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750178    5817 flags.go:64] FLAG: --port="10250"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750182    5817 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750187    5817 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0f43ad8e9cae1c23f"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750191    5817 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750194    5817 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750197    5817 flags.go:64] FLAG: --register-node="true"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750201    5817 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750205    5817 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750208    5817 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750211    5817 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:14:24.759020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750214    5817 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750218    5817 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750221    5817 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750224    5817 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750227    5817 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750230    5817 flags.go:64] FLAG: --runonce="false"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750233    5817 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750236    5817 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750239    5817 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750243    5817 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750250    5817 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750255    5817 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750260    5817 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750264    5817 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750269    5817 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750272    5817 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750275    5817 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750278    5817 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750281    5817 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750284    5817 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750287    5817 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750293    5817 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750296    5817 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750299    5817 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750304    5817 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:14:24.759754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750307    5817 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750310    5817 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750313    5817 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750316    5817 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750319    5817 flags.go:64] FLAG: --v="2"
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750324    5817 flags.go:64] FLAG: --version="false"
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750330    5817 flags.go:64] FLAG: --vmodule=""
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750337    5817 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750341    5817 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750517    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750522    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750525    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750528    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750531    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750535    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750539    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750541    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750546    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750549    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750553    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750555    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.760674 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750560    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750563    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750568    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750574    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750579    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750582    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750585    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750588    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750591    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750593    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750596    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750599    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750601    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750604    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750607    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750610    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750613    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750616    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750618    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750620    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.761809 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750623    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750626    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750629    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750631    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750634    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750636    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750639    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750643    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750649    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750654    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750658    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750660    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750665    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750667    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750670    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750673    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750675    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750678    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750680    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750683    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.762447 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750686    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750688    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750691    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750693    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750696    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750698    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750701    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750704    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750707    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750709    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750712    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750714    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750717    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750721    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750725    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750730    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750733    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750736    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750739    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.762988 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750741    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750745    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750748    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750751    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750754    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750758    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750760    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750763    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750766    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750768    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750771    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750774    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750776    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750778    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750781    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750784    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750787    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750789    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750792    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.763622 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750794    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750798    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750803    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750807    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750811    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750814    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750816    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750819    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750821    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750824    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750826    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750829    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750831    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750834    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750838    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750841    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.764440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.750868    5817 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750910    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750915    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750920    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750922    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750925    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750928    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750930    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750934    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750936    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750939    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750942    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750944    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750947    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750949    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750953    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750957    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750961    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750966    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.765046 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750970    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750972    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750975    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750977    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750980    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750983    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750985    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750988    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750991    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750993    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.750996    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751000    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751003    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751006    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751010    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751014    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751018    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751021    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751024    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.765977 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751028    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751032    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751037    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751042    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751047    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751052    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751055    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751058    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751061    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751063    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751066    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751069    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751072    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751074    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751077    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751079    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751082    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751085    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751089    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.766849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751092    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751094    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751097    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751099    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751102    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751106    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751110    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751114    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751119    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751123    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751128    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751131    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751134    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751136    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751139    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751142    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751145    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751148    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751150    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751153    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.768009 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751155    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751158    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751160    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751163    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751166    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751168    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751171    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751174    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751176    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751179    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751182    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751184    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751188    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751192    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751196    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751201    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751205    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751209    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751212    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.769051 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751214    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751217    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751219    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751224    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751227    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751230    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751233    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751236    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751238    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751241    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.751243    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.756047    5817 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.756063    5817 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.756081    5817 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.756087    5817 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756145    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756151    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756154    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756157    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.769965 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756160    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756163    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756166    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756169    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756172    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756174    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756177    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756180    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756182    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756185    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756188    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756190    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756193    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756196    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756199    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756201    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756205    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756209    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.770645 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756213    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756216    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756219    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756222    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756230    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756233    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756236    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756239    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756241    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756244    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756246    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756249    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756252    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756255    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756257    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756260    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756263    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756266    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756268    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756271    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.771314 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756273    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756276    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756279    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756282    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756285    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756288    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756291    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756294    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756296    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756299    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756302    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756304    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756308    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756310    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756313    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756315    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756318    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756320    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756323    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.772085 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756326    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756328    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756331    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756333    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756336    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756338    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756341    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756358    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756362    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756367    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756370    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756373    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756376    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756379    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756381    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756384    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756387    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756389    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756393    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.772600 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756395    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756398    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756401    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756403    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756406    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756409    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756411    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756414    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756416    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756420    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756425    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756428    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756430    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756433    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756436    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756438    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756441    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756444    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756446    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756449    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.773136 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756451    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756454    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756457    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756459    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756462    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756464    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.756503    5817 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756645    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756651    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756654    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756657    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756660    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756663    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756665    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756668    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756670    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.773777 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756673    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756676    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756679    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756682    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756684    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756687    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756690    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756693    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756695    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756698    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756700    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756703    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756705    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756707    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756710    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756713    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756715    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756718    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756720    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.774208 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756723    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756727    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756730    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756733    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756735    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756738    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756740    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756743    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756746    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756748    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756752    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756754    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756757    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756760    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756762    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756765    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756768    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756770    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756773    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.774774 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756776    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756778    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756781    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756783    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756786    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756789    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756791    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756794    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756796    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756799    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756801    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756804    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756807    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756810    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756813    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756816    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756819    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756821    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756824    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756826    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.775263 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756829    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756831    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756834    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756836    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756839    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756842    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756845    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756847    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756850    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756853    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756856    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756858    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756860    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756863    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756867    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756871    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756874    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756877    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756879    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756882    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.775864 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756885    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756887    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756889    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756895    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756898    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756900    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756903    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756906    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756909    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756911    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756914    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756917    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756919    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756922    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756926    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756929    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756932    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756935    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.776430 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756938    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.756961    5817 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756992    5817 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756996    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.756999    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757002    5817 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757005    5817 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757007    5817 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757010    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757012    5817 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757015    5817 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757018    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757021    5817 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757023    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757026    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:14:24.776926 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757029    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757031    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757034    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757038    5817 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757041    5817 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757045    5817 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757048    5817 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757051    5817 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757054    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757056    5817 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757059    5817 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757062    5817 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757064    5817 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757068    5817 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757071    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757073    5817 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757076    5817 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757080    5817 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757083    5817 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:14:24.777317 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757085    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757088    5817 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757091    5817 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757094    5817 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757096    5817 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757099    5817 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757102    5817 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757104    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757107    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757109    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757112    5817 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757115    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757117    5817 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757120    5817 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757122    5817 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757125    5817 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757129    5817 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757131    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757134    5817 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757137    5817 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:14:24.777857 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757139    5817 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757142    5817 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757145    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757147    5817 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757150    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757152    5817 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757157    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757159    5817 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757162    5817 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757165    5817 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757167    5817 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757170    5817 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757173    5817 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757176    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757179    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757182    5817 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757184    5817 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757187    5817 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757191    5817 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757194    5817 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:14:24.778404 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757197    5817 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757200    5817 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757203    5817 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757206    5817 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757209    5817 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757211    5817 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757214    5817 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757217    5817 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757221    5817 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757224    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757226    5817 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757229    5817 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757232    5817 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757234    5817 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757237    5817 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757240    5817 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757242    5817 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757245    5817 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757249    5817 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757252    5817 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:14:24.778970 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757255    5817 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757257    5817 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757260    5817 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757262    5817 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757265    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757268    5817 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757270    5817 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757273    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757276    5817 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757278    5817 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757281    5817 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757283    5817 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757286    5817 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:14:24.757288    5817 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.757338    5817 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.758314    5817 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.758385    5817 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.758565    5817 server.go:1014] "Starting client certificate rotation"
Oct 30 21:14:24.779670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.758702    5817 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.758755    5817 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:08:09 +0000 UTC" deadline="2028-07-11 17:33:14.52560837 +0000 UTC"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.758793    5817 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14876h18m49.766818594s"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.759002    5817 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.760178    5817 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.761276    5817 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.763363    5817 log.go:25] "Validated CRI v1 image API"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.764108    5817 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:14:24.780219 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.767567    5817 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a6346f91-3283-40aa-9ded-be1489083030:/dev/nvme0n1p4 d2b4df69-363c-46e0-9b51-e57e7d665682:/dev/nvme0n1p3]
Oct 30 21:14:24.780621 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.767596    5817 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/031196f0b61968e9538c1bc59c11f5c289c07f89daa348feb77d77928fd789c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/031196f0b61968e9538c1bc59c11f5c289c07f89daa348feb77d77928fd789c5/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/07dee1aaf5e446463af3845d15f7f105c9a5c164cc2f61808360e276361f7d74/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/07dee1aaf5e446463af3845d15f7f105c9a5c164cc2f61808360e276361f7d74/userdata/shm major:0 minor:264 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/099f62e01b20b261bc3dc27df9cef129812d4a1427f8343ddfab1aa32607e654/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/099f62e01b20b261bc3dc27df9cef129812d4a1427f8343ddfab1aa32607e654/userdata/shm major:0 minor:292 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1c65cc69476edcc37bc1a9315675780f94dd8657b3e1e95cc3a2e833f9300904/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1c65cc69476edcc37bc1a9315675780f94dd8657b3e1e95cc3a2e833f9300904/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3a9f6bf2a7992482cac94bb210b0a2f6e7b60ef2963c40f5c63d44d699fee3de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3a9f6bf2a7992482cac94bb210b0a2f6e7b60ef2963c40f5c63d44d699fee3de/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a/userdata/shm major:0 minor:344 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/53fcd4b90605a624857b017ef226e0d4d3eadd6aaacb51708e0abbb27390ac0e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/53fcd4b90605a624857b017ef226e0d4d3eadd6aaacb51708e0abbb27390ac0e/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6/userdata/shm major:0 minor:479 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7a28c672f9841339384d23fcb433779c2d474ac16076e87d31b9117a51aa74f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7a28c672f9841339384d23fcb433779c2d474ac16076e87d31b9117a51aa74f0/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a7e09fcc58f0a0598db52e432a5d10cb378dc4ef12669316847608262c54e92/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a7e09fcc58f0a0598db52e432a5d10cb378dc4ef12669316847608262c54e92/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a51533899aab92cfcb369664a6e255b21dce681e158bc0e20b552faf24813b8a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a51533899aab92cfcb369664a6e255b21dce681e158bc0e20b552faf24813b8a/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a9e0e935ea6c827ed9a7ac5df6e0c16f0eb8c438dd6bded6ac6de57f6e7dad72/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a9e0e935ea6c827ed9a7ac5df6e0c16f0eb8c438dd6bded6ac6de57f6e7dad72/userdata/shm major:0 minor:69 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ae96de185b3a643941046293f739122676d59ab6ab70968dffea95ac225650ec/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ae96de185b3a643941046293f739122676d59ab6ab70968dffea95ac225650ec/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c3f1549025938864c1d9ce58ad78a46679d596daf883de0e1739e34deb35f66a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c3f1549025938864c1d9ce58ad78a46679d596daf883de0e1739e34deb35f66a/userdata/shm major:0 minor:367 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c8aed7a3fa9d29a172172477906bbd396d25e2c7fe16ce3c113f2350c6dbc362/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c8aed7a3fa9d29a172172477906bbd396d25e2c7fe16ce3c113f2350c6dbc362/userdata/shm major:0 minor:369 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cecdfc44da39e62a81290b51a6a4019c80b17bf6496bff3b7780ff1435db71c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cecdfc44da39e62a81290b51a6a4019c80b17bf6496bff3b7780ff1435db71c1/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d542cf155686da3099789ecb97a85ed998892431c28cd1c34e0716b9f0820fbe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d542cf155686da3099789ecb97a85ed998892431c28cd1c34e0716b9f0820fbe/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d8635e6f3ade4b2e186e11dfff94108a77e85a4d877ae920b4ce2e818395340a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d8635e6f3ade4b2e186e11dfff94108a77e85a4d877ae920b4ce2e818395340a/userdata/shm major:0 minor:389 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d89ce32aa46061b2561ae609b3da7f02d8b8d42efa56d6afa0dbafa82b91c85a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d89ce32aa46061b2561ae609b3da7f02d8b8d42efa56d6afa0dbafa82b91c85a/userdata/shm major:0 minor:383 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e59bd031a441c1b7049a66d6e0392e6b82a3eac0e1965d2b1a18236bb71f2f70/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e59bd031a441c1b7049a66d6e0392e6b82a3eac0e1965d2b1a18236bb71f2f70/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f2539c35340f8647da0db7956f12ea396b4b245eec3644251d13031915940de4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f2539c35340f8647da0db7956f12ea396b4b245eec3644251d13031915940de4/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f945fdec6e7d13e452db60d6f5c4a70841659083f0f1b5a5045c8154ba0b04a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f945fdec6e7d13e452db60d6f5c4a70841659083f0f1b5a5045c8154ba0b04a7/userdata/shm major:0 minor:365 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f9d1f190488187e7cefad540ed43247a7ef98cca6a821bbb247a59e273609a5d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f9d1f190488187e7cefad540ed43247a7ef98cca6a821bbb247a59e273609a5d/userdata/shm major:0 minor:372 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fd642d2cf0fec17fd573570b7c747543289634a87bbe54d3fb539d79fe8dd3e0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fd642d2cf0fec17fd573570b7c747543289634a87bbe54d3fb539d79fe8dd3e0/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/alertmanager/9 major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/config-reloader/6 major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~empty-dir/config-out major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~projected/kube-api-access-cxqrw:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~projected/kube-api-access-cxqrw major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~projected/tls-assets major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/config-volume major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/web-config major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~projected/kube-api-access-kl2hj:{mountpoint:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~projected/kube-api-access-kl2hj major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:359 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2619e7f0-cd04-4aaf-8543-5db1718c5b57/volumes/kubernetes.io~projected/kube-api-access-4xg7f:{mountpoint:/var/lib/kubelet/pods/2619e7f0-cd04-4aaf-8543-5db1718c5b57/volumes/kubernetes.io~projected/kube-api-access-4xg7f major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2e048855-3db9-47b4-a1dc-29ee474f05e0/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/2e048855-3db9-47b4-a1dc-29ee474f05e0/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/45e2f746-f082-4371-b1c0-375a66cf81da/volumes/kubernetes.io~projected/kube-api-access-vc4x2:{mountpoint:/var/lib/kubelet/pods/45e2f746-f082-4371-b1c0-375a66cf81da/volumes/kubernetes.io~projected/kube-api-access-vc4x2 major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49b86000-eb20-4c99-9bfe-7269ea162576/volumes/kubernetes.io~secret/tls-certificates:{mountpoint:/var/lib/kubelet/pods/49b86000-eb20-4c99-9bfe-7269ea162576/volumes/kubernetes.io~secret/tls-certificates major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4ecf5604-81f2-4892-85a4-b2ac4dacf993/volumes/kubernetes.io~projected/kube-api-access-k9bhq:{mountpoint:/var/lib/kubelet/pods/4ecf5604-81f2-4892-85a4-b2ac4dacf993/volumes/kubernetes.io~projected/kube-api-access-k9bhq major:0 minor:382 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14/volumes/kubernetes.io~projected/kube-api-access-jz2dz:{mountpoint:/var/lib/kubelet/pods/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14/volumes/kubernetes.io~projected/kube-api-access-jz2dz major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5c33c14f-d505-434b-a45d-514cc39522c5/volumes/kubernetes.io~projected/kube-api-access-wpz2z:{mountpoint:/var/lib/kubelet/pods/5c33c14f-d505-434b-a45d-514cc39522c5/volumes/kubernetes.io~projected/kube-api-access-wpz2z major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~projected/kube-api-access-k8z9b:{mountpoint:/var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~projected/kube-api-access-k8z9b major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:467 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~empty-dir/config-out major:0 minor:465 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~projected/kube-api-access-8zrlw:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~projected/kube-api-access-8zrlw major:0 minor:469 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~projected/tls-assets major:0 minor:477 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/config major:0 minor:470 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:466 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:476 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:478 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:464 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:474 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:468 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:475 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/web-config major:0 minor:467 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/73b98cf0-3a8a-46b3-ac91-f18817011ee8/volumes/kubernetes.io~projected/kube-api-access-4dfl4:{mountpoint:/var/lib/kubelet/pods/73b98cf0-3a8a-46b3-ac91-f18817011ee8/volumes/kubernetes.io~projected/kube-api-access-4dfl4 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/73b98cf0-3a8a-46b3-ac91-f18817011ee8/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/73b98cf0-3a8a-46b3-ac91-f18817011ee8/volumes/kubernetes.io~secret/metrics-certs major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~projected/kube-api-access-rqndf:{mountpoint:/var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~projected/kube-api-access-rqndf major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/830286ce-0d57-4cb7-8f13-a298b08bd319/volumes/kubernetes.io~projected/kube-api-access-w9nsw:{mountpoint:/var/lib/kubelet/pods/830286ce-0d57-4cb7-8f13-a298b08bd319/volumes/kubernetes.io~projected/kube-api-access-w9nsw major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/830286ce-0d57-4cb7-8f13-a298b08bd319/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/830286ce-0d57-4cb7-8f13-a298b08bd319/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7/volumes/kubernetes.io~projected/kube-api-access-667fc:{mountpoint:/var/lib/kubelet/pods/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7/volumes/kubernetes.io~projected/kube-api-access-667fc major:0 minor:388 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:387 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volumes/kubernetes.io~projected/kube-api-access-8dr7r:{mountpoint:/var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volumes/kubernetes.io~projected/kube-api-access-8dr7r major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a12607d3-3f19-4743-807d-8e8356051388/volumes/kubernetes.io~projected/kube-api-access-74q97:{mountpoint:/var/lib/kubelet/pods/a12607d3-3f19-4743-807d-8e8356051388/volumes/kubernetes.io~projected/kube-api-access-74q97 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~projected/kube-api-access-l9qjj:{mountpoint:/var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~projected/kube-api-access-l9qjj major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c6c54779-d916-481f-b659-55e1b3630b77/volumes/kubernetes.io~projected/kube-api-access-6k6cd:{mountpoint:/var/lib/kubelet/pods/c6c54779-d916-481f-b659-55e1b3630b77/volumes/kubernetes.io~projected/kube-api-access-6k6cd major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c6c54779-d916-481f-b659-55e1b3630b77/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/c6c54779-d916-481f-b659-55e1b3630b77/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce211496-a484-4fc1-bf17-3a926847d855/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ce211496-a484-4fc1-bf17-3a926847d855/volumes/kubernetes.io~secret/agent-certs major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb/volumes/kubernetes.io~projected/kube-api-access-clwrj:{mountpoint:/var/lib/kubelet/pods/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb/volumes/kubernetes.io~projected/kube-api-access-clwrj major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e60d4632-5192-4ce4-9190-36ca9849be8a/volumes/kubernetes.io~projected/kube-api-access-5pvs8:{mountpoint:/var/lib/kubelet/pods/e60d4632-5192-4ce4-9190-36ca9849be8a/volumes/kubernetes.io~projected/kube-api-access-5pvs8 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~projected/kube-api-access-rc49h:{mountpoint:/var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~projected/kube-api-access-rc49h major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f24e3266-f0e8-4610-ab07-0448f4537d25/volumes/kubernetes.io~projected/kube-api-access-222vk:{mountpoint:/var/lib/kubelet/pods/f24e3266-f0e8-4610-ab07-0448f4537d25/volumes/kubernetes.io~projected/kube-api-access-222vk major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~projected/kube-api-access-n82bm:{mountpoint:/var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~projected/kube-api-access-n82bm major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:250 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/5dc12ff15fd7bc261b89cfb46d24b98b37ddde645a30cf83256f058a2dc219c7/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/a2ec4676c0fc133e698aae869c3ac82301dd8b6c6611ff62e858299c46fefb61/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/b3b86af23d803bcf7075b8ac89020f22973697d66307515e5187d0cb4a9d1c12/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/d5f4eb3725095d54b7b5f3b43a5e2df2eaebf0eb8d8a027edfeec29b2e690851/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/f6f4334f1421f3a592e51a51d45e95ef4ed1d03b414f8a19f352130f2998b759/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/701df9d0765e900270551109a144e5a368ca5364bf8abe337b30063162c9ce2e/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/b3f4cd936cd9515bfb0bd2a698adcb577837510ad2c34dc39176e394e95de530/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/df35f61270906ae6c3d369d697d78813805ad89c832358e3c34063b38fd598eb/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/bf985f9df40fb42537e6de7898039e66974f9dd5c7c61eee243472dcfb981a73/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/7bd2ee90659e97bfaef0d9b63d3d4589640c1c2b1cdae10751f1f37bd1fba0c1/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/6825d49014b62dbd4ba078026620f388394559b447b0e282cd997880ce232c85/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/6571d5a37c5ceb012ad48c3c65d50f46022f3d34eb45bb9dfd384e35b58caf85/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/4d0bbaf46397fc8840047a72129d506cce10d75be2960a161d07c36ec39fbe00/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/0b6465c786b16386e7c11f341b1bb9c7e25b13aff29288b67bb2e3867fc6e831/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/cbb665de96bb5cfc844e349e295d539d21303390e9853d292ecfcdbae33721f2/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/16c1d029fb3e28057599e98528376d4bde6d1691444445c7cf0feffb06bd60c3/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/7b6f0ddce7566a8f7963046bf48e8b16be7a79920b5313591d8ee42ac12c9d7e/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/41879e06990a54bded5a34e1ee858e5efc04bdfe163f4b58750bdfb7a829011f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/a54965ba50a29b2f5324049289fe440e2918d3bc1f084ae8a6cc86f757ae97e0/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/0d7d00a6661b9c3697ab961932c6d4c1a294a4a238a283c7be16a6e40084317a/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/670fffa558f601e9ea10c2382dbb5fde62f57ff5ff9d32c36bad6443be764ee9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/97184619710d219bea54f27aae10f9b3e5b6c57be26bd4bc5271efdedfa95286/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/823a0115bed560db8c464e9217a5f6fc13dec4baaa76aad3de561a5f9356d6c8/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/be69a1c2a96f0d178d1c9837b0def26c93d89772df1d268e66cbb9a544be1f81/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/e0cc61232eb90e23b6b148e40e08c83467539f265db1ae45b5417115294873ed/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-224:{mountpoint:/var/lib/containers/storage/overlay/241f3c44e61ad3192b19e36e9dcdeb54ee2304437bdff9fa25a37143fc54ec70/merged major:0 minor:224 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/475965976be6a93f0ac0dc9c6e504f8690440eb11517852942d383828614c8f3/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/20881b8bf34d2033608dc4131b258e96ec4d5379653ba87bba3f85595309b503/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/ddf71ed80e63d3685a177e75ff0e57ec26919a9b57736350ceba97682362801d/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/14239d73d04312033085eab8926b9a9e32a6b2d59cac6ed871b6042c407a8d9d/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/8fcdb81a29233f3bc547726ad0ec27ccf599c867822bff1322894c9dac4676b9/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/927a65ca8a2376752ff8da9c9a0ccca1091013f52cdefe2070d3f409c3280578/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/e40b74629386f8c684682ed582b158401432878a907bf1cffbd4e13659dbf4de/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/09ec455b6191117f258ca1b6ff02da0ddaea280fa7dcb3e16508dddc7aa1a7fc/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/9a12519fbd79805c3e292b6c7cd87ed9effc2653d318a85f5925f20ec38faf4b/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/22c6d5bfcd0ec9c892be024b1e590b1e7282cb9266d7f385c7a07d2dc12410aa/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/122ff26e2a229fc93f4d0af88053209e8c13053e1ecfc669491aecb9f09eaa04/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/b286da4a8b0b519663500dafdf07be12ff33bbce0750616db2d2a1b11310b3d5/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/00128172a1ff810afde6941404baabbb7efc44f348e2286d4ef7d12419f5d688/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/119759e4f29de66aa84c94cf7077b829fb6740b086d3449b253531c2ddcb731a/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/4db2e8b0ff01ebda226ef486ae29302aa81d9f0286bc9e64e00064664ba69d03/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-313:{mountpoint:/var/lib/containers/storage/overlay/8240c0882146823989955cf6061d62696a4bb17d144a5bb48e7539a345ba5474/merged major:0 minor:313 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/8b3842c3f557e2f28e28f5383ef8833c0bee3f8600b8e1f4e50876d4003bbe2e/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/f45d2f1252253e4288eb55504d0794052fe6ff64e31cba96bc485a3a9e98c67d/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/acd92e923fcee12ccb8f7cb0cbc4501a53bd191acaa1b3dc67ac3219842d8dd6/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/39918c808ca5578db38d99a70282c9b56f90cd3de29fa33afe8cacd1410a03b1/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/19146e2a98948afb9559419b394d9897438e0dbcc83b7266e8fcee469d672d8a/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/28bd79443dfd08726626427410a20222a94d61f2b935ede5c462f424a202552e/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/d2243273cb761c880633de422625b79c97a9a7a44f0719fd8fc7c2d704d36a6d/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-378:{mountpoint:/var/lib/containers/storage/overlay/fcf3bd53856d60f22a13c2e05f0141b1bc0b8ae402fb11f6606b65f8c9737e7a/merged major:0 minor:378 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/f3ba6662bb68743daca039fbec9928d0b63f2ce49a5880551bfc868276bce5ed/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-385:{mountpoint:/var/lib/containers/storage/overlay/ba4b2082fd3c8aa68cf09ff7ab686d3d3d14284cdcee8026a381753ac5474331/merged major:0 minor:385 fsType:overlay blockSize:0} overlay_0-391:{mountpoint:/var/lib/containers/storage/overlay/9a46883a02285af22afcff422a78be1a8940777f7a2fc48f3f5acf61a3c47327/merged major:0 minor:391 fsType:overlay blockSize:0} overlay_0-393:{mountpoint:/var/lib/containers/storage/overlay/f011b8e520a1e8bb674adcb64028fe18465bc9b118618dfe90c10f708ed73202/merged major:0 minor:393 fsType:overlay blockSize:0} overlay_0-395:{mountpoint:/var/lib/containers/storage/overlay/394f89c6214489e885e601aaaa8170926b8bebdb07cd2355da6b00f69d75ca95/merged major:0 minor:395 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/0c480c3d35f6c0c0a4eb4ab72b82db5f52f121ebcdea13b742be22ba75f65329/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-406:{mountpoint:/var/lib/containers/storage/overlay/a84aaaf4c5ac9e464dadd8e429b94a416e60b8e3ca70acf90035625ad934e1c2/merged major:0 minor:406 fsType:overlay blockSize:0} overlay_0-412:{mountpoint:/var/lib/containers/storage/overlay/3df720e0d5f24e0f0e58b42466e2a6e82a4d66a8dda5996bf2a0a864be1058cc/merged major:0 minor:412 fsType:overlay blockSize:0} overlay_0-417:{mountpoint:/var/lib/containers/storage/overlay/4b00ab34e8b1b283d00c4bcbbfc8456687c7da95c438968581344efddb0105f3/merged major:0 minor:417 fsType:overlay blockSize:0} overlay_0-422:{mountpoint:/var/lib/containers/storage/overlay/58c2b00ec71fe97d46fb8cfee7d6fda298b1c1eabbea3d9a037052f16440d959/merged major:0 minor:422 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/b3e960bee6275dd6fe7ee5b60debb37f439105dffe660807c70fdcd276d0d9f5/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-434:{mountpoint:/var/lib/containers/storage/overlay/472bab7dcecf40d6c55ac05f82dc0eb81599ecebb525b6dc2963da345fabcbce/merged major:0 minor:434 fsType:overlay blockSize:0} overlay_0-439:{mountpoint:/var/lib/containers/storage/overlay/7118d3b520b184f9302dd2a0851da40b4ae90063ff5d19e7a7936eab08934b5f/merged major:0 minor:439 fsType:overlay blockSize:0} overlay_0-444:{mountpoint:/var/lib/containers/storage/overlay/8385ccc1fd01378cd2a4f51b59ad6e5aea1528f8aac3d8efb01fb2fc14f5d018/merged major:0 minor:444 fsType:overlay blockSize:0} overlay_0-449:{mountpoint:/var/lib/containers/storage/overlay/51481ccc60285157e4ceb8f27aca77c37fc6b60a03802d24dbcd6f44ee86d07e/merged major:0 minor:449 fsType:overlay blockSize:0} overlay_0-454:{mountpoint:/var/lib/containers/storage/overlay/9aa0593c73797dfc266e2cc85574f46b58a90cf797ba45df2bea7fd9cb735cfd/merged major:0 minor:454 fsType:overlay blockSize:0} overlay_0-459:{mountpoint:/var/lib/containers/storage/overlay/d695d4d8c21f6256a2d4760d25c62341cf79835b8498ee75e4f95d651592059c/merged major:0 minor:459 fsType:overlay blockSize:0} overlay_0-481:{mountpoint:/var/lib/containers/storage/overlay/ae0fc62142161ebc543feca82d17bc844c1b835339abba6936f2744551d86656/merged major:0 minor:481 fsType:overlay blockSize:0} overlay_0-483:{mountpoint:/var/lib/containers/storage/overlay/83fa8b51657303aa09444e46aa59d80e82536f6a13394def77a60961edac3e34/merged major:0 minor:483 fsType:overlay blockSize:0} overlay_0-489:{mountpoint:/var/lib/containers/storage/overlay/95a88a9dee7daae02a868398dcf076db49fdcb899cc13f1c915379a5c73d22a8/merged major:0 minor:489 fsType:overlay blockSize:0} overlay_0-494:{mountpoint:/var/lib/containers/storage/overlay/79091c415b0166d02f323bc46ed7e3983d121d99a8635f99a8347af8f3fccc28/merged major:0 minor:494 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/118999acd7c30bb01fbdcc7c821ad32d2c53e03f9c29764a327f4853354b967d/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/44dd70edaaea1932f7f382d86ece37719232338780c0778b8f7e106b00e89c6c/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/43361cc0f11377fa44887a782478c5beb1897598783efdc9bf17af2332f755a4/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/3424485ddbcace4079afac2bfbed6f84e805da273bec650e63c1985de9b1edeb/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/3e350196add4326f4fcc88eb9ece5255707a338411ec0f2c1a7eae2155be1932/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/b6163df1e7720be1384e8bb1cf6afffff7253f84e8cd019e18e7fd529d4ab224/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/a5d2018b9067075a1d3c23423975b492d135289d48d5f3ce797c18b7158b3f48/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/e835e0587d2511478ee135d2f9bb35473f188e75fef27e5643c0e91b87c11b85/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/acb73c31a02092333180877ce8f8d6830bc3d5323293c869d880c86a35ae1491/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/297a0c1362c53a49aed739e9413c50d139e31225853def996f856719184db455/merged major:0 minor:97 fsType:overlay blockSize:0}]
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.777903    5817 manager.go:217] Machine: {Timestamp:2026-10-30 21:14:24.777446474 +0000 UTC m=+0.065169010 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101595 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ddd251d4a914d15d0559882a967be SystemUUID:ec2ddd25-1d4a-914d-15d0-559882a967be BootID:1967fb36-9ba8-4672-9851-852186d557d7 Filesystems:[{Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~projected/kube-api-access-rqndf DeviceMajor:0 DeviceMinor:286 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:321 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:478 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e60d4632-5192-4ce4-9190-36ca9849be8a/volumes/kubernetes.io~projected/kube-api-access-5pvs8 DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c6c54779-d916-481f-b659-55e1b3630b77/volumes/kubernetes.io~projected/kube-api-access-6k6cd DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:317 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/830286ce-0d57-4cb7-8f13-a298b08bd319/volumes/kubernetes.io~projected/kube-api-access-w9nsw DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~projected/kube-api-access-rc49h DeviceMajor:0 DeviceMinor:281 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-434 DeviceMajor:0 DeviceMinor:434 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:466 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/ce211496-a484-4fc1-bf17-3a926847d855/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/45e2f746-f082-4371-b1c0-375a66cf81da/volumes/kubernetes.io~projected/kube-api-access-vc4x2 DeviceMajor:0 DeviceMinor:371 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-395 DeviceMajor:0 DeviceMinor:395 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/830286ce-0d57-4cb7-8f13-a298b08bd319/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e59bd031a441c1b7049a66d6e0392e6b82a3eac0e1965d2b1a18236bb71f2f70/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-439 DeviceMajor:0 DeviceMinor:439 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:476 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/53fcd4b90605a624857b017ef226e0d4d3eadd6aaacb51708e0abbb27390ac0e/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f945fdec6e7d13e452db60d6f5c4a70841659083f0f1b5a5045c8154ba0b04a7/userdata/shm DeviceMajor:0 DeviceMinor:365 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/07dee1aaf5e446463af3845d15f7f105c9a5c164cc2f61808360e276361f7d74/userdata/shm DeviceMajor:0 DeviceMinor:264 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:284 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-385 DeviceMajor:0 DeviceMinor:385 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fd642d2cf0fec17fd573570b7c747543289634a87bbe54d3fb539d79fe8dd3e0/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~projected/kube-api-access-l9qjj DeviceMajor:0 DeviceMinor:287 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:319 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~projected/kube-api-access-cxqrw DeviceMajor:0 DeviceMinor:330 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:351 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:324 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:357 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-459 DeviceMajor:0 DeviceMinor:459 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~projected/kube-api-access-8zrlw DeviceMajor:0 DeviceMinor:469 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-489 DeviceMajor:0 DeviceMinor:489 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4ecf5604-81f2-4892-85a4-b2ac4dacf993/volumes/kubernetes.io~projected/kube-api-access-k9bhq DeviceMajor:0 DeviceMinor:382 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:251 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:475 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2619e7f0-cd04-4aaf-8543-5db1718c5b57/volumes/kubernetes.io~projected/kube-api-access-4xg7f DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a7e09fcc58f0a0598db52e432a5d10cb378dc4ef12669316847608262c54e92/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:283 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:470 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6/userdata/shm DeviceMajor:0 DeviceMinor:479 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/73b98cf0-3a8a-46b3-ac91-f18817011ee8/volumes/kubernetes.io~projected/kube-api-access-4dfl4 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-224 DeviceMajor:0 DeviceMinor:224 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:325 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:467 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c6c54779-d916-481f-b659-55e1b3630b77/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/807ebbbd-72a2-47c3-b50a-2b044f71a8ff/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:285 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~projected/kube-api-access-kl2hj DeviceMajor:0 DeviceMinor:360 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-417 DeviceMajor:0 DeviceMinor:417 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:465 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/031196f0b61968e9538c1bc59c11f5c289c07f89daa348feb77d77928fd789c5/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~projected/kube-api-access-n82bm DeviceMajor:0 DeviceMinor:252 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c8aed7a3fa9d29a172172477906bbd396d25e2c7fe16ce3c113f2350c6dbc362/userdata/shm DeviceMajor:0 DeviceMinor:369 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a99b0768-fe06-4d64-95cb-64f5334596f8/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:282 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d8635e6f3ade4b2e186e11dfff94108a77e85a4d877ae920b4ce2e818395340a/userdata/shm DeviceMajor:0 DeviceMinor:389 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:477 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:280 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:318 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:321 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-449 DeviceMajor:0 DeviceMinor:449 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-378 DeviceMajor:0 DeviceMinor:378 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-481 DeviceMajor:0 DeviceMinor:481 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a12607d3-3f19-4743-807d-8e8356051388/volumes/kubernetes.io~projected/kube-api-access-74q97 DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~projected/kube-api-access-k8z9b DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-393 DeviceMajor:0 DeviceMinor:393 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-422 DeviceMajor:0 DeviceMinor:422 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-454 DeviceMajor:0 DeviceMinor:454 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5c33c14f-d505-434b-a45d-514cc39522c5/volumes/kubernetes.io~projected/kube-api-access-wpz2z DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9f6d30dd-2479-494f-af1c-dec5361950d9/volumes/kubernetes.io~projected/kube-api-access-8dr7r DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7a28c672f9841339384d23fcb433779c2d474ac16076e87d31b9117a51aa74f0/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/49b86000-eb20-4c99-9bfe-7269ea162576/volumes/kubernetes.io~secret/tls-certificates DeviceMajor:0 DeviceMinor:219 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:356 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cecdfc44da39e62a81290b51a6a4019c80b17bf6496bff3b7780ff1435db71c1/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ff1ba61a-eb5d-4b06-8690-b564a17d47fd/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:250 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/edbff937-2134-4112-8ff4-3c50ed16ffe2/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:276 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:322 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a/userdata/shm DeviceMajor:0 DeviceMinor:344 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-391 DeviceMajor:0 DeviceMinor:391 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-406 DeviceMajor:0 DeviceMinor:406 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:474 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:467 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-494 DeviceMajor:0 DeviceMinor:494 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c3f1549025938864c1d9ce58ad78a46679d596daf883de0e1739e34deb35f66a/userdata/shm DeviceMajor:0 DeviceMinor:367 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-412 DeviceMajor:0 DeviceMinor:412 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:359 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/73b98cf0-3a8a-46b3-ac91-f18817011ee8/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:361 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5d4a4e1b-e451-434f-91f4-b35024c1e4c0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ae96de185b3a643941046293f739122676d59ab6ab70968dffea95ac225650ec/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:387 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/099f62e01b20b261bc3dc27df9cef129812d4a1427f8343ddfab1aa32607e654/userdata/shm DeviceMajor:0 DeviceMinor:292 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7/volumes/kubernetes.io~projected/kube-api-access-667fc DeviceMajor:0 DeviceMinor:388 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:464 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-483 DeviceMajor:0 DeviceMinor:483 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f24e3266-f0e8-4610-ab07-0448f4537d25/volumes/kubernetes.io~projected/kube-api-access-222vk DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb/volumes/kubernetes.io~projected/kube-api-access-clwrj DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:320 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-444 DeviceMajor:0 DeviceMinor:444 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:468 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1c65cc69476edcc37bc1a9315675780f94dd8657b3e1e95cc3a2e833f9300904/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f2539c35340f8647da0db7956f12ea396b4b245eec3644251d13031915940de4/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14/volumes/kubernetes.io~projected/kube-api-access-jz2dz DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a51533899aab92cfcb369664a6e255b21dce681e158bc0e20b552faf24813b8a/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3a9f6bf2a7992482cac94bb210b0a2f6e7b60ef2963c40f5c63d44d699fee3de/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-313 DeviceMajor:0 DeviceMinor:313 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:355 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d542cf155686da3099789ecb97a85ed998892431c28cd1c34e0716b9f0820fbe/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a9e0e935ea6c827ed9a7ac5df6e0c16f0eb8c438dd6bded6ac6de57f6e7dad72/userdata/shm DeviceMajor:0 DeviceMinor:69 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:323 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bacc5cd-6d42-48f6-8807-5117835ca615/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:358 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2e048855-3db9-47b4-a1dc-29ee474f05e0/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:362 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f9d1f190488187e7cefad540ed43247a7ef98cca6a821bbb247a59e273609a5d/userdata/shm DeviceMajor:0 DeviceMinor:372 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d89ce32aa46061b2561ae609b3da7f02d8b8d42efa56d6afa0dbafa82b91c85a/userdata/shm DeviceMajor:0 DeviceMinor:383 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:07dee1aaf5e4464 MacAddress:96:22:48:87:ec:8a Speed:10000 Mtu:8901} {Name:099f62e01b20b26 MacAddress:e6:0f:91:11:7a:cb Speed:10000 Mtu:8901} {Name:3a9f6bf2a799248 MacAddress:66:7a:70:66:cf:77 Speed:10000 Mtu:8901} {Name:44ac13a354faa5b MacAddress:82:44:57:42:ab:05 Speed:10000 Mtu:8901} {Name:53fcd4b90605a62 MacAddress:06:ce:e6:87:d4:69 Speed:10000 Mtu:8901} {Name:64acd159955eee1 MacAddress:56:40:55:43:a2:09 Speed:10000 Mtu:8901} {Name:a51533899aab92c MacAddress:c2:00:7d:26:ed:3c Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:d3:40:ff:3f Speed:0 Mtu:9001} {Name:br-int MacAddress:1e:e3:b6:56:f4:a8 Speed:0 Mtu:8901} {Name:c3f154902593886 MacAddress:5a:cf:7d:73:55:3d Speed:10000 Mtu:8901} {Name:c8aed7a3fa9d29a MacAddress:aa:46:77:a6:c1:2d Speed:10000 Mtu:8901} {Name:d8635e6f3ade4b2 MacAddress:e2:b7:fd:cb:75:f8 Speed:10000 Mtu:8901} {Name:d89ce32aa46061b MacAddress:72:6e:a3:46:aa:f9 Speed:10000 Mtu:8901} {Name:e59bd031a441c1b MacAddress:06:f0:ba:4a:ed:e7 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:d3:40:ff:3f Speed:0 Mtu:9001} {Name:f945fdec6e7d13e MacAddress:62:2b:ca:8c:db:a5 Speed:10000 Mtu:8901} {Name:f9d1f190488187e MacAddress:b2:a9:a8:ea:d4:a5 Speed:10000 Mtu:8901} {Name:fd642d2cf0fec17 MacAddress:7a:39:64:6e:28:40 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:36:eb:9f:0b:3e:14 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:1e:ea:69:53:7c:75 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778361    5817 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778415    5817 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778679    5817 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778698    5817 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-25.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778816    5817 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778824    5817 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778834    5817 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778847    5817 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.778936    5817 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.779084    5817 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.779182    5817 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.779214    5817 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.779222    5817 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.779232    5817 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.779242    5817 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.779250    5817 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.780184    5817 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.780195    5817 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:14:24.781184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.780675    5817 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:14:24.796269 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796247    5817 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:14:24.796269 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796263    5817 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:14:24.796417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796374    5817 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:14:24.796570 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796559    5817 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:14:24.796606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796588    5817 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:14:24.796684 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796675    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:14:24.796724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796692    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:14:24.796724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796702    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:14:24.796724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796718    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:14:24.796724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796723    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796730    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796736    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796742    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796752    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796773    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796794    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796804    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796834    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:14:24.796849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.796840    5817 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:14:24.800220 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.800205    5817 server.go:1263] "Started kubelet"
Oct 30 21:14:24.800310 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.800283    5817 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:14:24.800991 ip-10-0-142-25 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:14:24.803863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.801222    5817 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:14:24.803863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.801313    5817 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:14:24.803863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.802796    5817 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.805224    5817 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.805507    5817 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.805517    5817 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.805521    5817 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:24.805590    5817 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.805632    5817 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.808531    5817 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:08:11 +0000 UTC" deadline="2028-04-09 09:33:02.79428167 +0000 UTC"
Oct 30 21:14:24.810333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.808578    5817 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12636h18m37.985709979s"
Oct 30 21:14:24.811119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811096    5817 factory.go:55] Registering systemd factory
Oct 30 21:14:24.811232 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811124    5817 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:14:24.811524 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811469    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811530    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53cd7b3b-32e9-4fed-9b76-a16d9ca69d14" volumeName="kubernetes.io/configmap/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811550    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811566    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a99b0768-fe06-4d64-95cb-64f5334596f8" volumeName="kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811583    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edbff937-2134-4112-8ff4-3c50ed16ffe2" volumeName="kubernetes.io/configmap/edbff937-2134-4112-8ff4-3c50ed16ffe2-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811599    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811616    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811653    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53cd7b3b-32e9-4fed-9b76-a16d9ca69d14" volumeName="kubernetes.io/empty-dir/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-data-volume" seLinuxMountContext=""
Oct 30 21:14:24.811677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811672    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811686    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811699    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811712    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811737    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edbff937-2134-4112-8ff4-3c50ed16ffe2" volumeName="kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811750    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811765    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/empty-dir/0bacc5cd-6d42-48f6-8807-5117835ca615-tmp" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811776    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecf5604-81f2-4892-85a4-b2ac4dacf993" volumeName="kubernetes.io/projected/4ecf5604-81f2-4892-85a4-b2ac4dacf993-kube-api-access-k9bhq" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811788    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d4a4e1b-e451-434f-91f4-b35024c1e4c0" volumeName="kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-tuned" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811799    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" volumeName="kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811804    5817 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811811    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" volumeName="kubernetes.io/empty-dir/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-textfile" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811857    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/projected/0bacc5cd-6d42-48f6-8807-5117835ca615-kube-api-access-kl2hj" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811869    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811909    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f6d30dd-2479-494f-af1c-dec5361950d9" volumeName="kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811924    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f6d30dd-2479-494f-af1c-dec5361950d9" volumeName="kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-env-overrides" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811939    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a99b0768-fe06-4d64-95cb-64f5334596f8" volumeName="kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.811995    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f24e3266-f0e8-4610-ab07-0448f4537d25" volumeName="kubernetes.io/projected/f24e3266-f0e8-4610-ab07-0448f4537d25-kube-api-access-222vk" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812013    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812032    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e048855-3db9-47b4-a1dc-29ee474f05e0" volumeName="kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812048    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812106    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53cd7b3b-32e9-4fed-9b76-a16d9ca69d14" volumeName="kubernetes.io/secret/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812125    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812145    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7" volumeName="kubernetes.io/projected/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-kube-api-access-667fc" seLinuxMountContext=""
Oct 30 21:14:24.812169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812163    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a99b0768-fe06-4d64-95cb-64f5334596f8" volumeName="kubernetes.io/projected/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-api-access-l9qjj" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812197    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d295e58b-95c4-4dbd-9f04-8c8e2c760dbb" volumeName="kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812215    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812233    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/configmap/0bacc5cd-6d42-48f6-8807-5117835ca615-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812250    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="73b98cf0-3a8a-46b3-ac91-f18817011ee8" volumeName="kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812268    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a99b0768-fe06-4d64-95cb-64f5334596f8" volumeName="kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812286    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a99b0768-fe06-4d64-95cb-64f5334596f8" volumeName="kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812305    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce211496-a484-4fc1-bf17-3a926847d855" volumeName="kubernetes.io/secret/ce211496-a484-4fc1-bf17-3a926847d855-agent-certs" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812321    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812340    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812404    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812421    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" volumeName="kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812436    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7" volumeName="kubernetes.io/secret/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812453    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f6d30dd-2479-494f-af1c-dec5361950d9" volumeName="kubernetes.io/secret/9f6d30dd-2479-494f-af1c-dec5361950d9-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812464    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812482    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2619e7f0-cd04-4aaf-8543-5db1718c5b57" volumeName="kubernetes.io/configmap/2619e7f0-cd04-4aaf-8543-5db1718c5b57-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812496    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2619e7f0-cd04-4aaf-8543-5db1718c5b57" volumeName="kubernetes.io/projected/2619e7f0-cd04-4aaf-8543-5db1718c5b57-kube-api-access-4xg7f" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812509    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4ecf5604-81f2-4892-85a4-b2ac4dacf993" volumeName="kubernetes.io/empty-dir/4ecf5604-81f2-4892-85a4-b2ac4dacf993-tmp" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812527    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5c33c14f-d505-434b-a45d-514cc39522c5" volumeName="kubernetes.io/projected/5c33c14f-d505-434b-a45d-514cc39522c5-kube-api-access-wpz2z" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812541    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d4a4e1b-e451-434f-91f4-b35024c1e4c0" volumeName="kubernetes.io/projected/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-kube-api-access-k8z9b" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812554    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edbff937-2134-4112-8ff4-3c50ed16ffe2" volumeName="kubernetes.io/projected/edbff937-2134-4112-8ff4-3c50ed16ffe2-kube-api-access-rc49h" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812572    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff1ba61a-eb5d-4b06-8690-b564a17d47fd" volumeName="kubernetes.io/configmap/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812586    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812600    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812614    5817 factory.go:153] Registering CRI-O factory
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812637    5817 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812702    5817 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812620    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812725    5817 factory.go:103] Registering Raw factory
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812744    5817 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812787    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812847    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812871    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" volumeName="kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-tls" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812926    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff1ba61a-eb5d-4b06-8690-b564a17d47fd" volumeName="kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812945    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53cd7b3b-32e9-4fed-9b76-a16d9ca69d14" volumeName="kubernetes.io/projected/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-api-access-jz2dz" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812962    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d4a4e1b-e451-434f-91f4-b35024c1e4c0" volumeName="kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-tmp" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812981    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.812997    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="73b98cf0-3a8a-46b3-ac91-f18817011ee8" volumeName="kubernetes.io/projected/73b98cf0-3a8a-46b3-ac91-f18817011ee8-kube-api-access-4dfl4" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813077    5817 manager.go:319] Starting recovery of all containers
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813081    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" volumeName="kubernetes.io/projected/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-kube-api-access-rqndf" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813102    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce211496-a484-4fc1-bf17-3a926847d855" volumeName="kubernetes.io/configmap/ce211496-a484-4fc1-bf17-3a926847d855-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813120    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e60d4632-5192-4ce4-9190-36ca9849be8a" volumeName="kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813140    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813163    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49b86000-eb20-4c99-9bfe-7269ea162576" volumeName="kubernetes.io/secret/49b86000-eb20-4c99-9bfe-7269ea162576-tls-certificates" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813271    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813302    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="830286ce-0d57-4cb7-8f13-a298b08bd319" volumeName="kubernetes.io/empty-dir/830286ce-0d57-4cb7-8f13-a298b08bd319-tmp-dir" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813320    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e60d4632-5192-4ce4-9190-36ca9849be8a" volumeName="kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813366    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="45e2f746-f082-4371-b1c0-375a66cf81da" volumeName="kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813385    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813403    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 21:14:24.813432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813448    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f6d30dd-2479-494f-af1c-dec5361950d9" volumeName="kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-config" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813467    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f6d30dd-2479-494f-af1c-dec5361950d9" volumeName="kubernetes.io/projected/9f6d30dd-2479-494f-af1c-dec5361950d9-kube-api-access-8dr7r" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813484    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a12607d3-3f19-4743-807d-8e8356051388" volumeName="kubernetes.io/projected/a12607d3-3f19-4743-807d-8e8356051388-kube-api-access-74q97" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813498    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d295e58b-95c4-4dbd-9f04-8c8e2c760dbb" volumeName="kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813510    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813522    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813534    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813547    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813558    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a99b0768-fe06-4d64-95cb-64f5334596f8" volumeName="kubernetes.io/empty-dir/a99b0768-fe06-4d64-95cb-64f5334596f8-volume-directive-shadow" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813570    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d295e58b-95c4-4dbd-9f04-8c8e2c760dbb" volumeName="kubernetes.io/projected/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-kube-api-access-clwrj" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813583    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edbff937-2134-4112-8ff4-3c50ed16ffe2" volumeName="kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813595    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff1ba61a-eb5d-4b06-8690-b564a17d47fd" volumeName="kubernetes.io/projected/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-kube-api-access-n82bm" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813606    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813620    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813632    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813644    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813662    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="830286ce-0d57-4cb7-8f13-a298b08bd319" volumeName="kubernetes.io/projected/830286ce-0d57-4cb7-8f13-a298b08bd319-kube-api-access-w9nsw" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813674    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e60d4632-5192-4ce4-9190-36ca9849be8a" volumeName="kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813699    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e60d4632-5192-4ce4-9190-36ca9849be8a" volumeName="kubernetes.io/projected/e60d4632-5192-4ce4-9190-36ca9849be8a-kube-api-access-5pvs8" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813712    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff1ba61a-eb5d-4b06-8690-b564a17d47fd" volumeName="kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813723    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" volumeName="kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813736    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bacc5cd-6d42-48f6-8807-5117835ca615" volumeName="kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813759    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61b3c88c-4121-4a5b-acb2-8d7e78a71243" volumeName="kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813773    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" volumeName="kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813785    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="830286ce-0d57-4cb7-8f13-a298b08bd319" volumeName="kubernetes.io/configmap/830286ce-0d57-4cb7-8f13-a298b08bd319-config-volume" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813800    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="830286ce-0d57-4cb7-8f13-a298b08bd319" volumeName="kubernetes.io/secret/830286ce-0d57-4cb7-8f13-a298b08bd319-metrics-tls" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813811    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a12607d3-3f19-4743-807d-8e8356051388" volumeName="kubernetes.io/empty-dir/a12607d3-3f19-4743-807d-8e8356051388-tmp-dir" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813823    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6c54779-d916-481f-b659-55e1b3630b77" volumeName="kubernetes.io/secret/c6c54779-d916-481f-b659-55e1b3630b77-cert" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813835    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5c33c14f-d505-434b-a45d-514cc39522c5" volumeName="kubernetes.io/configmap/5c33c14f-d505-434b-a45d-514cc39522c5-serviceca" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813847    5817 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6c54779-d916-481f-b659-55e1b3630b77" volumeName="kubernetes.io/projected/c6c54779-d916-481f-b659-55e1b3630b77-kube-api-access-6k6cd" seLinuxMountContext=""
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813858    5817 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:14:24.816042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.813865    5817 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:14:24.833664 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.833630    5817 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:14:24.835277 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.835248    5817 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:14:24.835277 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.835276    5817 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:14:24.835493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.835298    5817 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:14:24.835493 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:24.835336    5817 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:14:24.858976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.858946    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016" exitCode=0
Oct 30 21:14:24.874262 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.874228    5817 generic.go:356] "Generic (PLEG): container finished" podID="807ebbbd-72a2-47c3-b50a-2b044f71a8ff" containerID="843ac632d5ef41bd8c1425988fa256358df9800b8b2d79b3910737f54a05d44a" exitCode=0
Oct 30 21:14:24.890267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.890233    5817 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="262218a4d13282700bbc586605219999fc5ae084031d2c9342445e77a469c5d4" exitCode=0
Oct 30 21:14:24.890267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.890266    5817 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="23e79ba76906ad165f9a6d9076c5c98edde98cc454549d28398c20718fd42d3f" exitCode=0
Oct 30 21:14:24.890267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.890277    5817 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="f05fa59705456101284dc9f5baf0cf3e3c355e4b905638749ee765ac21101a8b" exitCode=0
Oct 30 21:14:24.890571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.890289    5817 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="f5253043fe816611ce6874f00cb04f44cc93c76d2e79684614699a38e970bd74" exitCode=0
Oct 30 21:14:24.890571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.890300    5817 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="cbb46554dd4c6dee9bcfd65f66e94e3910e5f947e8d37c83aa1e728ccb34dd7a" exitCode=0
Oct 30 21:14:24.890571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.890311    5817 generic.go:356] "Generic (PLEG): container finished" podID="e60d4632-5192-4ce4-9190-36ca9849be8a" containerID="1ab00859156a5ebff3e719bc42b4baf5887322fc4f4190d93607e70ff2826bf0" exitCode=0
Oct 30 21:14:24.896983 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.896957    5817 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244" exitCode=0
Oct 30 21:14:24.905806 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:24.905782    5817 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-25.us-west-2.compute.internal\" not found"
Oct 30 21:14:24.917439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.917337    5817 generic.go:356] "Generic (PLEG): container finished" podID="121e9c4b72ca7d02f36ba35f5de497be" containerID="dc2dbfb7f92acd1f86e0e19430cc439a62404c68ceb9ab92b9798ef8b91d2b89" exitCode=0
Oct 30 21:14:24.935606 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:24.935576    5817 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:14:24.951920 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.951898    5817 manager.go:324] Recovery completed
Oct 30 21:14:24.981018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.979401    5817 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:14:24.981321 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.981301    5817 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:14:24.981472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.981453    5817 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:14:24.981682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.981667    5817 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:14:24.982193 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.982169    5817 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:14:24.982293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.982211    5817 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:14:24.982293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.982225    5817 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.985695    5817 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.985710    5817 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.985732    5817 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.985881    5817 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.985896    5817 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986004    5817 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986012    5817 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986020    5817 policy_none.go:50] "Start"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986033    5817 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986045    5817 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986209    5817 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986218    5817 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:14:24.986751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.986226    5817 policy_none.go:44] "Start"
Oct 30 21:14:24.991270 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.991240    5817 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:14:24.993737 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.993723    5817 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:14:24.994207 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:24.994188    5817 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:14:24.994207 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.994207    5817 server.go:87] "Starting device plugin registration server"
Oct 30 21:14:24.994523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.994509    5817 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:14:24.994599 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.994524    5817 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:14:24.994654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.994625    5817 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:14:24.995251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.994724    5817 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:14:24.995335 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.995260    5817 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:14:24.995335 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.995267    5817 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:14:24.995473 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.995335    5817 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:14:24.995249432Z","UUID":"0f9a2117-2456-4f3f-bb9f-de2d924201e5","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:14:24.996830 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.996800    5817 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:14:24.996924 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:24.996853    5817 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:14:24.997554 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:24.997532    5817 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:14:25.028400 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.028312    5817 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:14:25.075751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.075726    5817 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:14:25.095175 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.095144    5817 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:14:25.096189 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.096172    5817 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:14:25.096293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.096203    5817 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:14:25.096293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.096214    5817 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-25.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:14:25.096293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.096271    5817 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.136311 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.136259    5817 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"]
Oct 30 21:14:25.137031 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.137016    5817 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.137097 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.137050    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"30e0ff2135e8ccc515bab6d6f7a12d83","Type":"ContainerStarted","Data":"eff6cd1f6f82d0d503e8135c02e77c967f300ea21a8e8fc22e170d3974dc8652"}
Oct 30 21:14:25.137097 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.137090    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"30e0ff2135e8ccc515bab6d6f7a12d83","Type":"ContainerStarted","Data":"d0717555e79e99c9739a0c618db95d069bfe714f7eaf18d386c8d26e013a41a8"}
Oct 30 21:14:25.137204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.137123    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"121e9c4b72ca7d02f36ba35f5de497be","Type":"ContainerStarted","Data":"75f2f2f151b5897effe08f338bc9a9a6cf1f49bc287d1075baf9c799ca39431c"}
Oct 30 21:14:25.137204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.137140    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"121e9c4b72ca7d02f36ba35f5de497be","Type":"ContainerDied","Data":"dc2dbfb7f92acd1f86e0e19430cc439a62404c68ceb9ab92b9798ef8b91d2b89"}
Oct 30 21:14:25.137204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.137168    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal" event={"ID":"121e9c4b72ca7d02f36ba35f5de497be","Type":"ContainerStarted","Data":"94e09cf10e92c1aea45c15c4d40d5ec3ea1d81a06861496a33264c3db285cf41"}
Oct 30 21:14:25.137316 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.137286    5817 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.197723 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.197638    5817 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:14:25.197723 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:25.197691    5817 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.198250 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.198232    5817 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:14:25.198292 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:25.198272    5817 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.204191 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.204172    5817 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.204305 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.204238    5817 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.284141 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.284107    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.284315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.284146    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.284315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.284172    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/30e0ff2135e8ccc515bab6d6f7a12d83-config\") pod \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"30e0ff2135e8ccc515bab6d6f7a12d83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.384950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.384913    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.384950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.384955    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.385176 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.384979    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/30e0ff2135e8ccc515bab6d6f7a12d83-config\") pod \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"30e0ff2135e8ccc515bab6d6f7a12d83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.385176 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.385024    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/30e0ff2135e8ccc515bab6d6f7a12d83-config\") pod \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"30e0ff2135e8ccc515bab6d6f7a12d83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.385176 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.385025    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.385176 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.385029    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/121e9c4b72ca7d02f36ba35f5de497be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" (UID: \"121e9c4b72ca7d02f36ba35f5de497be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.780043 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.780008    5817 apiserver.go:52] "Watching apiserver"
Oct 30 21:14:25.925858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.925826    5817 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:14:25.926569 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.926543    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0","openshift-monitoring/node-exporter-t47rj","openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv","openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5","openshift-network-diagnostics/network-check-target-jg5k9","kube-system/konnectivity-agent-d4w2x","kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal","openshift-image-registry/node-ca-6prr8","openshift-monitoring/thanos-querier-5c84f565c4-h97ps","openshift-dns/node-resolver-z7qz6","openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59","openshift-console/downloads-6dd789c8b7-npz4c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-vlddw","kube-system/global-pull-secret-syncer-26g9z","openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk","openshift-multus/multus-qxmhx","openshift-monitoring/prometheus-k8s-0","openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh","openshift-ovn-kubernetes/ovnkube-node-nlctd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk","openshift-multus/multus-additional-cni-plugins-b94vg","openshift-multus/network-metrics-daemon-fxrjh","openshift-network-operator/iptables-alerter-blkqp","openshift-insights/insights-runtime-extractor-8vhcn","openshift-dns/dns-default-8fpfq","openshift-ingress-canary/ingress-canary-q2p4x"]
Oct 30 21:14:25.932339 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.932311    5817 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.935478 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.935441    5817 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:25.978619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978588    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:14:25.978619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978613    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978588    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978644    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978653    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ph6zc\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978645    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978623    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978700    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978711    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978752    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qzjzq\""
Oct 30 21:14:25.978884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.978852    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5fn5j\""
Oct 30 21:14:25.979222 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.979014    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:14:25.979222 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.979128    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w5d85\""
Oct 30 21:14:25.979300 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.979233    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sqpcg\""
Oct 30 21:14:25.979337 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.979310    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:14:25.979410 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:25.979370    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:14:26.009114 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.009084    5817 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:14:26.027910 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.027875    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:14:26.027910 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.027880    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:14:26.028092 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.027931    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.028092 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.027939    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.028407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028393    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.028534 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028515    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:14:26.028594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028518    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:14:26.028594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028523    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:14:26.028594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028554    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:14:26.028706 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028617    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:14:26.028852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028830    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.028852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028837    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.029033 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.028891    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:14:26.029103 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029079    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 21:14:26.029103 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029084    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029108    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029118    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029130    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029143    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029185    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-dpfbq\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029189    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029193    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.029206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029209    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f9wbl\""
Oct 30 21:14:26.029547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029215    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:14:26.029547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029231    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:14:26.029547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029196    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.029547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029246    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.029547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029281    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.029547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029365    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.029547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029491    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x6zwr\""
Oct 30 21:14:26.029804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029564    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.029804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029697    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-lm9pr\""
Oct 30 21:14:26.029804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029756    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 21:14:26.029917 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029808    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:14:26.029917 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029810    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.029917 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029844    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-79sqh\""
Oct 30 21:14:26.030040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029929    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:14:26.030040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.029955    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4v9xp\""
Oct 30 21:14:26.030040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.030031    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:14:26.030268 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.030225    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:14:26.030268 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.030266    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:14:26.030563 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.030551    5817 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:14:26.030619 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:26.030591    5817 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:26.030665 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.030614    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:14:26.074406 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074378    5817 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:14:26.074556 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:26.074440    5817 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-25.us-west-2.compute.internal"
Oct 30 21:14:26.074672 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074652    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:14:26.074747 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074678    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:14:26.074747 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074705    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kvf2h\""
Oct 30 21:14:26.074852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074765    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:14:26.074852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074788    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:14:26.074852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074825    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:14:26.074995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074879    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:14:26.074995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074892    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:14:26.074995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074901    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q4s27\""
Oct 30 21:14:26.074995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.074912    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:14:26.075151 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075002    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:14:26.075151 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075070    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-47rmpbtts3l4q\""
Oct 30 21:14:26.075391 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075342    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:14:26.075513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075442    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h6tkb\""
Oct 30 21:14:26.075513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075448    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:14:26.075908 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075886    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:14:26.076020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075936    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:14:26.076020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075939    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-j7gnd\""
Oct 30 21:14:26.076020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075969    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:14:26.076182 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075495    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k9bmv\""
Oct 30 21:14:26.076288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.075497    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:14:26.076288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.076213    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:14:26.076288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.076279    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xx59j\""
Oct 30 21:14:26.076570 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.076538    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-p88cx\""
Oct 30 21:14:26.076825 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.076809    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b9xqh\""
Oct 30 21:14:26.077124 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.077094    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwnd5\""
Oct 30 21:14:26.077598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.077567    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:14:26.077598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.077588    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xhltr\""
Oct 30 21:14:26.077805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.077789    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:14:26.078088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.078067    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:14:26.078180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.078104    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:14:26.078249 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.078232    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:14:26.078318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.078272    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:14:26.078318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.078314    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.078474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.078460    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w5xnx\""
Oct 30 21:14:26.078632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.078619    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:14:26.079106 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079091    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-68m835m929agv\""
Oct 30 21:14:26.079212 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079196    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.079293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079242    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-5mkrp\""
Oct 30 21:14:26.079399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079338    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-m55dr\""
Oct 30 21:14:26.079492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079437    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:14:26.079492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079450    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:14:26.079492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079370    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:14:26.079685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079663    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:14:26.079741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079706    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:14:26.079813 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079787    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:14:26.079875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079855    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:14:26.079951 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079908    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:14:26.080026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.079974    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h52wm\""
Oct 30 21:14:26.080026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.080002    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.080140 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.080121    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:14:26.080210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.080161    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:14:26.080259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.080243    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:14:26.080259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.080255    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:14:26.080636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.080618    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:14:26.088586 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088564    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.088687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088600    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-conf-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.088687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088625    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dfl4\" (UniqueName: \"kubernetes.io/projected/73b98cf0-3a8a-46b3-ac91-f18817011ee8-kube-api-access-4dfl4\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:26.088687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088654    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.088687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088682    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.088891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088709    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k6cd\" (UniqueName: \"kubernetes.io/projected/c6c54779-d916-481f-b659-55e1b3630b77-kube-api-access-6k6cd\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:14:26.088891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088735    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-kubernetes\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.088891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088842    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-conf\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.089032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088892    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.089032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088916    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.089032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.088998    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-run\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.089032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089005    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.089032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089019    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089047    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12607d3-3f19-4743-807d-8e8356051388-tmp-dir\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089065    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089123    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089163    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12607d3-3f19-4743-807d-8e8356051388-tmp-dir\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089213    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089264    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089298    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089389    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089449    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.089472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089470    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089560    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-etc-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089584    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089662    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-netd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089676    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089695    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6d30dd-2479-494f-af1c-dec5361950d9-ovn-node-metrics-cert\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089717    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49b86000-eb20-4c99-9bfe-7269ea162576-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-l4g59\" (UID: \"49b86000-eb20-4c99-9bfe-7269ea162576\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089739    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089765    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/830286ce-0d57-4cb7-8f13-a298b08bd319-config-volume\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.089816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089788    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c33c14f-d505-434b-a45d-514cc39522c5-serviceca\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:14:26.090071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089820    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9qjj\" (UniqueName: \"kubernetes.io/projected/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-api-access-l9qjj\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.090071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089845    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.090071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089861    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.090071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089871    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.090071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089897    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.090071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089921    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f6d30dd-2479-494f-af1c-dec5361950d9-ovn-node-metrics-cert\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.090071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.089959    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49b86000-eb20-4c99-9bfe-7269ea162576-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-l4g59\" (UID: \"49b86000-eb20-4c99-9bfe-7269ea162576\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090069    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090137    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/830286ce-0d57-4cb7-8f13-a298b08bd319-config-volume\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090193    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090221    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-system-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090226    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090236    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c54779-d916-481f-b659-55e1b3630b77-cert\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090265    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-modprobe-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090282    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-env-overrides\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090298    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9nsw\" (UniqueName: \"kubernetes.io/projected/830286ce-0d57-4cb7-8f13-a298b08bd319-kube-api-access-w9nsw\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090318    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090362    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090388    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-var-lib-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090480    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-env-overrides\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090490    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6c54779-d916-481f-b659-55e1b3630b77-cert\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090520    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c33c14f-d505-434b-a45d-514cc39522c5-serviceca\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:14:26.090554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090541    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-bin\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090579    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-multus\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090574    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090601    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090607    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce211496-a484-4fc1-bf17-3a926847d855-agent-certs\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090655    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090689    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-socket-dir-parent\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090713    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-hostroot\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090715    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e048855-3db9-47b4-a1dc-29ee474f05e0-original-pull-secret\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090737    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-lib-modules\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090761    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-var-lib-kubelet\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090785    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-crio-socket\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090809    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090788    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce211496-a484-4fc1-bf17-3a926847d855-agent-certs\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090835    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090933    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pvs8\" (UniqueName: \"kubernetes.io/projected/e60d4632-5192-4ce4-9190-36ca9849be8a-kube-api-access-5pvs8\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.090959    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091021    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091038    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2dz\" (UniqueName: \"kubernetes.io/projected/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-api-access-jz2dz\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091054    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091070    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091085    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.091090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091099    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091105    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cni-binary-copy\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091140    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/830286ce-0d57-4cb7-8f13-a298b08bd319-metrics-tls\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091166    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-tuned\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091139    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091202    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-registration-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091258    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091277    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-tuned\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091307    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-sys-fs\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091339    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cni-binary-copy\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091340    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/830286ce-0d57-4cb7-8f13-a298b08bd319-metrics-tls\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091362    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-data-volume\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091407    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091414    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091433    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cnibin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091438    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-data-volume\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091465    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-kubelet\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091485    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-multus-certs\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091518    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-metrics-client-ca\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091539    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091545    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091568    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091689    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-metrics-client-ca\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091715    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091740    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-config\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091756    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zrlw\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091827    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-sys\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091839    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091870    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-host\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091909    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-grpc-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.091946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091943    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-node-log\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091974    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091975    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-config\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.091968    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092016    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092036    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092050    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c33c14f-d505-434b-a45d-514cc39522c5-host\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092070    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092141    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-grpc-tls\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092167    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9bhq\" (UniqueName: \"kubernetes.io/projected/4ecf5604-81f2-4892-85a4-b2ac4dacf993-kube-api-access-k9bhq\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092184    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092198    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092213    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830286ce-0d57-4cb7-8f13-a298b08bd319-tmp-dir\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092236    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqndf\" (UniqueName: \"kubernetes.io/projected/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-kube-api-access-rqndf\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092265    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz2z\" (UniqueName: \"kubernetes.io/projected/5c33c14f-d505-434b-a45d-514cc39522c5-kube-api-access-wpz2z\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092279    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092293    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092327    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092368    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-script-lib\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092377    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830286ce-0d57-4cb7-8f13-a298b08bd319-tmp-dir\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092394    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-sys\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092419    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092420    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092301    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092496    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092523    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0bacc5cd-6d42-48f6-8807-5117835ca615-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092555    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092559    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n82bm\" (UniqueName: \"kubernetes.io/projected/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-kube-api-access-n82bm\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092586    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092613    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092622    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f6d30dd-2479-494f-af1c-dec5361950d9-ovnkube-script-lib\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092641    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092670    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092703    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqrw\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092734    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-root\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092762    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092787    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092804    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092804    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092875    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092888    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092971    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecf5604-81f2-4892-85a4-b2ac4dacf993-tmp\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.092993    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-systemd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093014    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093022    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2hj\" (UniqueName: \"kubernetes.io/projected/0bacc5cd-6d42-48f6-8807-5117835ca615-kube-api-access-kl2hj\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093042    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ecf5604-81f2-4892-85a4-b2ac4dacf993-tmp\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093123    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-ovn\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093140    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093155    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bacc5cd-6d42-48f6-8807-5117835ca615-metrics-client-ca\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093169    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-slash\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093228    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.093261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093275    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093294    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-device-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093313    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-netns\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093336    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-log-socket\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093369    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0bacc5cd-6d42-48f6-8807-5117835ca615-metrics-client-ca\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093405    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbff937-2134-4112-8ff4-3c50ed16ffe2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093454    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clwrj\" (UniqueName: \"kubernetes.io/projected/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-kube-api-access-clwrj\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093495    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73b98cf0-3a8a-46b3-ac91-f18817011ee8-metrics-certs\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093523    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-667fc\" (UniqueName: \"kubernetes.io/projected/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-kube-api-access-667fc\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093583    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbff937-2134-4112-8ff4-3c50ed16ffe2-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093604    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093707    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093751    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-netns\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093835    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e60d4632-5192-4ce4-9190-36ca9849be8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093841    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-etc-kubernetes\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093873    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093914    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093916    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093954    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce211496-a484-4fc1-bf17-3a926847d855-konnectivity-ca\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.093973    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-kubelet\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094004    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094007    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc49h\" (UniqueName: \"kubernetes.io/projected/edbff937-2134-4112-8ff4-3c50ed16ffe2-kube-api-access-rc49h\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094107    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-wtmp\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094115    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce211496-a484-4fc1-bf17-3a926847d855-konnectivity-ca\") pod \"konnectivity-agent-d4w2x\" (UID: \"ce211496-a484-4fc1-bf17-3a926847d855\") " pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094133    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094137    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-tmp\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094165    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8z9b\" (UniqueName: \"kubernetes.io/projected/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-kube-api-access-k8z9b\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094195    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a99b0768-fe06-4d64-95cb-64f5334596f8-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094216    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2619e7f0-cd04-4aaf-8543-5db1718c5b57-iptables-alerter-script\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094232    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094198    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-tmp\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094255    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74q97\" (UniqueName: \"kubernetes.io/projected/a12607d3-3f19-4743-807d-8e8356051388-kube-api-access-74q97\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094257    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a99b0768-fe06-4d64-95cb-64f5334596f8-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094297    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-k8s-cni-cncf-io\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094323    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-cnibin\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094379    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-222vk\" (UniqueName: \"kubernetes.io/projected/f24e3266-f0e8-4610-ab07-0448f4537d25-kube-api-access-222vk\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094408    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094417    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2619e7f0-cd04-4aaf-8543-5db1718c5b57-iptables-alerter-script\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094514    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094537    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094558    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/edbff937-2134-4112-8ff4-3c50ed16ffe2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094593    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094703    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysconfig\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094715    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094777    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094804    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bacc5cd-6d42-48f6-8807-5117835ca615-tmp\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094823    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xg7f\" (UniqueName: \"kubernetes.io/projected/2619e7f0-cd04-4aaf-8543-5db1718c5b57-kube-api-access-4xg7f\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094837    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094853    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a12607d3-3f19-4743-807d-8e8356051388-hosts-file\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094887    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bacc5cd-6d42-48f6-8807-5117835ca615-tmp\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094906    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094925    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2619e7f0-cd04-4aaf-8543-5db1718c5b57-host-slash\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.094968    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095002    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a99b0768-fe06-4d64-95cb-64f5334596f8-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095010    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095035    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-daemon-config\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095051    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095070    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-systemd-units\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095048    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095101    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095135    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-bin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095153    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-textfile\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095169    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095173    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095190    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095215    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095235    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095236    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-daemon-config\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095246    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-textfile\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095250    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095314    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095367    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-kubelet-config\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095398    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dr7r\" (UniqueName: \"kubernetes.io/projected/9f6d30dd-2479-494f-af1c-dec5361950d9-kube-api-access-8dr7r\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095369    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095426    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095461    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095484    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095487    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-systemd\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095512    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-socket-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.095460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095488    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095551    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-dbus\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095594    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095614    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095641    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095651    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-tls\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095890    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-tls\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.095891    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.097745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.097723    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.144509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.144479    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dfl4\" (UniqueName: \"kubernetes.io/projected/73b98cf0-3a8a-46b3-ac91-f18817011ee8-kube-api-access-4dfl4\") pod \"network-metrics-daemon-fxrjh\" (UID: \"73b98cf0-3a8a-46b3-ac91-f18817011ee8\") " pod="openshift-multus/network-metrics-daemon-fxrjh"
Oct 30 21:14:26.144660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.144643    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k6cd\" (UniqueName: \"kubernetes.io/projected/c6c54779-d916-481f-b659-55e1b3630b77-kube-api-access-6k6cd\") pod \"ingress-canary-q2p4x\" (UID: \"c6c54779-d916-481f-b659-55e1b3630b77\") " pod="openshift-ingress-canary/ingress-canary-q2p4x"
Oct 30 21:14:26.145689 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.145673    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9nsw\" (UniqueName: \"kubernetes.io/projected/830286ce-0d57-4cb7-8f13-a298b08bd319-kube-api-access-w9nsw\") pod \"dns-default-8fpfq\" (UID: \"830286ce-0d57-4cb7-8f13-a298b08bd319\") " pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:26.145905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.145887    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9qjj\" (UniqueName: \"kubernetes.io/projected/a99b0768-fe06-4d64-95cb-64f5334596f8-kube-api-access-l9qjj\") pod \"kube-state-metrics-5dc8d7d89b-rr7fk\" (UID: \"a99b0768-fe06-4d64-95cb-64f5334596f8\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-rr7fk"
Oct 30 21:14:26.145944 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.145934    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc4x2\" (UniqueName: \"kubernetes.io/projected/45e2f746-f082-4371-b1c0-375a66cf81da-kube-api-access-vc4x2\") pod \"network-check-target-jg5k9\" (UID: \"45e2f746-f082-4371-b1c0-375a66cf81da\") " pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:26.176088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.176063    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pvs8\" (UniqueName: \"kubernetes.io/projected/e60d4632-5192-4ce4-9190-36ca9849be8a-kube-api-access-5pvs8\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.178143 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.178125    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zrlw\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\") pod \"prometheus-k8s-0\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:26.178545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.178530    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clwrj\" (UniqueName: \"kubernetes.io/projected/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-kube-api-access-clwrj\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.179659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.179643    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqrw\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\") pod \"alertmanager-main-0\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:14:26.179822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.179802    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n82bm\" (UniqueName: \"kubernetes.io/projected/ff1ba61a-eb5d-4b06-8690-b564a17d47fd-kube-api-access-n82bm\") pod \"prometheus-operator-9cd89b74c-nkmlv\" (UID: \"ff1ba61a-eb5d-4b06-8690-b564a17d47fd\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-nkmlv"
Oct 30 21:14:26.179886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.179870    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqndf\" (UniqueName: \"kubernetes.io/projected/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-kube-api-access-rqndf\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.180022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.180008    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9bhq\" (UniqueName: \"kubernetes.io/projected/4ecf5604-81f2-4892-85a4-b2ac4dacf993-kube-api-access-k9bhq\") pod \"downloads-6dd789c8b7-npz4c\" (UID: \"4ecf5604-81f2-4892-85a4-b2ac4dacf993\") " pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:26.183796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.183778    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-222vk\" (UniqueName: \"kubernetes.io/projected/f24e3266-f0e8-4610-ab07-0448f4537d25-kube-api-access-222vk\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.184152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.184120    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dr7r\" (UniqueName: \"kubernetes.io/projected/9f6d30dd-2479-494f-af1c-dec5361950d9-kube-api-access-8dr7r\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.184431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.184412    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2dz\" (UniqueName: \"kubernetes.io/projected/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-kube-api-access-jz2dz\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.184487 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.184452    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74q97\" (UniqueName: \"kubernetes.io/projected/a12607d3-3f19-4743-807d-8e8356051388-kube-api-access-74q97\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:14:26.184657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.184636    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc49h\" (UniqueName: \"kubernetes.io/projected/edbff937-2134-4112-8ff4-3c50ed16ffe2-kube-api-access-rc49h\") pod \"openshift-state-metrics-7bd7489957-vcnbh\" (UID: \"edbff937-2134-4112-8ff4-3c50ed16ffe2\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-vcnbh"
Oct 30 21:14:26.184782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.184767    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2hj\" (UniqueName: \"kubernetes.io/projected/0bacc5cd-6d42-48f6-8807-5117835ca615-kube-api-access-kl2hj\") pod \"thanos-querier-5c84f565c4-h97ps\" (UID: \"0bacc5cd-6d42-48f6-8807-5117835ca615\") " pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:26.185474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.185459    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpz2z\" (UniqueName: \"kubernetes.io/projected/5c33c14f-d505-434b-a45d-514cc39522c5-kube-api-access-wpz2z\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:14:26.186902 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.186885    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xg7f\" (UniqueName: \"kubernetes.io/projected/2619e7f0-cd04-4aaf-8543-5db1718c5b57-kube-api-access-4xg7f\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:14:26.188790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.188768    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-667fc\" (UniqueName: \"kubernetes.io/projected/85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7-kube-api-access-667fc\") pod \"monitoring-plugin-77f4f76594-gfbx5\" (UID: \"85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:26.194172 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.194155    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8z9b\" (UniqueName: \"kubernetes.io/projected/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-kube-api-access-k8z9b\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.196194 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196177    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-netns\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196202    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-etc-kubernetes\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196218    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-kubelet\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.196327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196239    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-wtmp\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.196327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196268    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-k8s-cni-cncf-io\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196274    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-netns\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196279    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-etc-kubernetes\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196310    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-k8s-cni-cncf-io\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196309    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-cnibin\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196325    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-node-exporter-wtmp\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196338    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysconfig\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196307    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-kubelet\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196331    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-cnibin\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196392    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysconfig\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196385    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a12607d3-3f19-4743-807d-8e8356051388-hosts-file\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196423    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2619e7f0-cd04-4aaf-8543-5db1718c5b57-host-slash\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196435    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a12607d3-3f19-4743-807d-8e8356051388-hosts-file\") pod \"node-resolver-z7qz6\" (UID: \"a12607d3-3f19-4743-807d-8e8356051388\") " pod="openshift-dns/node-resolver-z7qz6"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196448    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-systemd-units\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196464    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-bin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196480    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196495    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-kubelet-config\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196495    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2619e7f0-cd04-4aaf-8543-5db1718c5b57-host-slash\") pod \"iptables-alerter-blkqp\" (UID: \"2619e7f0-cd04-4aaf-8543-5db1718c5b57\") " pod="openshift-network-operator/iptables-alerter-blkqp"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196513    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-systemd\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196519    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-systemd-units\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196527    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-bin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196534    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-socket-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196544    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196549    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-dbus\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196573    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-systemd\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196553    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-kubelet-config\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196583    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-conf-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.196579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196587    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-socket-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196582    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e048855-3db9-47b4-a1dc-29ee474f05e0-dbus\") pod \"global-pull-secret-syncer-26g9z\" (UID: \"2e048855-3db9-47b4-a1dc-29ee474f05e0\") " pod="kube-system/global-pull-secret-syncer-26g9z"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196630    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-conf-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196674    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-kubernetes\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196690    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-conf\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196704    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-run\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196719    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196732    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-kubernetes\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196745    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196762    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-etc-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196769    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-run\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196779    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196802    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-etc-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196808    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196833    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-netd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196852    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196866    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196869    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-conf\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196876    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-netd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196889    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196883    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-system-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196910    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-system-cni-dir\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196930    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-modprobe-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196963    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-var-lib-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196972    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-modprobe-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196973    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-etc-sysctl-d\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.196989    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-bin\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197009    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-var-lib-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197013    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-multus\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197034    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-cni-bin\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197040    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-socket-dir-parent\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197058    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-cni-multus\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197066    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-hostroot\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197096    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-lib-modules\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197098    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-hostroot\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197110    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-var-lib-kubelet\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197131    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-multus-socket-dir-parent\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197142    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-crio-socket\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197162    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-var-lib-kubelet\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197168    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197174    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-lib-modules\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197191    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-registration-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197205    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-sys-fs\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197220    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cnibin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197234    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-kubelet\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197191    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/53cd7b3b-32e9-4fed-9b76-a16d9ca69d14-crio-socket\") pod \"insights-runtime-extractor-8vhcn\" (UID: \"53cd7b3b-32e9-4fed-9b76-a16d9ca69d14\") " pod="openshift-insights/insights-runtime-extractor-8vhcn"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197271    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-var-lib-kubelet\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197271    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-openvswitch\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197283    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-registration-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197295    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-multus-certs\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197302    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-cnibin\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197316    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197361    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e60d4632-5192-4ce4-9190-36ca9849be8a-system-cni-dir\") pod \"multus-additional-cni-plugins-b94vg\" (UID: \"e60d4632-5192-4ce4-9190-36ca9849be8a\") " pod="openshift-multus/multus-additional-cni-plugins-b94vg"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197368    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d295e58b-95c4-4dbd-9f04-8c8e2c760dbb-host-run-multus-certs\") pod \"multus-qxmhx\" (UID: \"d295e58b-95c4-4dbd-9f04-8c8e2c760dbb\") " pod="openshift-multus/multus-qxmhx"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197362    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-sys-fs\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197404    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-sys\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197417    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-host\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197437    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-node-log\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197468    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197493    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c33c14f-d505-434b-a45d-514cc39522c5-host\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197467    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-sys\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197503    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d4a4e1b-e451-434f-91f4-b35024c1e4c0-host\") pod \"tuned-vlddw\" (UID: \"5d4a4e1b-e451-434f-91f4-b35024c1e4c0\") " pod="openshift-cluster-node-tuning-operator/tuned-vlddw"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197497    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-node-log\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197520    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-sys\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197519    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197535    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.197506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197553    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-root\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197536    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c33c14f-d505-434b-a45d-514cc39522c5-host\") pod \"node-ca-6prr8\" (UID: \"5c33c14f-d505-434b-a45d-514cc39522c5\") " pod="openshift-image-registry/node-ca-6prr8"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197581    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-systemd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197591    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-root\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197595    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/807ebbbd-72a2-47c3-b50a-2b044f71a8ff-sys\") pod \"node-exporter-t47rj\" (UID: \"807ebbbd-72a2-47c3-b50a-2b044f71a8ff\") " pod="openshift-monitoring/node-exporter-t47rj"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197610    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-ovn\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197636    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-slash\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197641    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-etc-selinux\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197670    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-device-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197680    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-ovn\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197688    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-netns\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197701    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-run-systemd\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197704    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-log-socket\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197733    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-slash\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197734    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-host-run-netns\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197758    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f6d30dd-2479-494f-af1c-dec5361950d9-log-socket\") pod \"ovnkube-node-nlctd\" (UID: \"9f6d30dd-2479-494f-af1c-dec5361950d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:26.199417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.197758    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f24e3266-f0e8-4610-ab07-0448f4537d25-device-dir\") pod \"aws-ebs-csi-driver-node-xcnbk\" (UID: \"f24e3266-f0e8-4610-ab07-0448f4537d25\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xcnbk"
Oct 30 21:14:26.242653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:26.242631    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:14:27.395814 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:27.395776    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:27.406049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:27.406017    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c84f565c4-h97ps"
Oct 30 21:14:28.552473 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:28.552440    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8fpfq"
Oct 30 21:14:29.032395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:29.032341    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:14:29.637125 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:29.636013    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:30.070296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.069899    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:30.070296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.070132    5817 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:14:30.958057 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.958024    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerStarted","Data":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"}
Oct 30 21:14:30.958538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.958096    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerStarted","Data":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"}
Oct 30 21:14:30.958538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.958123    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerStarted","Data":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"}
Oct 30 21:14:30.958538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.958148    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerStarted","Data":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"}
Oct 30 21:14:30.958538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.958175    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerStarted","Data":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"}
Oct 30 21:14:30.958538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:30.958202    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerStarted","Data":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"}
Oct 30 21:14:31.480105 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:31.480068    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-l4g59"
Oct 30 21:14:31.812452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:31.812373    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:14:32.317081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:32.317042    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:32.320767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:32.320742    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-gfbx5"
Oct 30 21:14:44.017871 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:44.017828    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-npz4c" event={"ID":"4ecf5604-81f2-4892-85a4-b2ac4dacf993","Type":"ContainerStarted","Data":"484d064a46d423f3134b51ae262deaeb06b3ca96dcdb5bcbae227e0b87a329a4"}
Oct 30 21:14:44.018440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:44.018138    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:44.035670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:44.035636    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-npz4c"
Oct 30 21:14:45.101447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:45.101383    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nlctd"
Oct 30 21:14:46.345692 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:14:46.345644    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:14:52.670328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:52.670294    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d4w2x"
Oct 30 21:14:59.035201 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:14:59.035169    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jg5k9"
Oct 30 21:15:16.337973 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:16.337938    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:15:18.322913 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:18.322866    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:18.364642 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:18.364610    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:19.183937 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:19.183909    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:28.896159 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:28.896119    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:28.896692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:28.896547    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="alertmanager" containerID="cri-o://d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798" gracePeriod=120
Oct 30 21:15:28.896771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:28.896741    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-web" containerID="cri-o://d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620" gracePeriod=120
Oct 30 21:15:28.896826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:28.896767    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="prom-label-proxy" containerID="cri-o://b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7" gracePeriod=120
Oct 30 21:15:28.896826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:28.896716    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy" containerID="cri-o://b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e" gracePeriod=120
Oct 30 21:15:28.896928 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:28.896838    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="config-reloader" containerID="cri-o://f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d" gracePeriod=120
Oct 30 21:15:28.896928 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:28.896888    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-metric" containerID="cri-o://d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f" gracePeriod=120
Oct 30 21:15:29.155458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.155391    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.187096 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187064    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187116    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187141    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187166    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187185    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187203    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187227    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187280    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187327    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187386    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187422    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187454    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187486    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\") pod \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\" (UID: \"0a9adc37-ec78-4112-93c6-e8e02e34bfd2\") "
Oct 30 21:15:29.187930 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.187757    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:29.188557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.188265    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:29.188557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.188265    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:29.190492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.190465    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:29.190808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.190778    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:29.190906 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.190793    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "kube-api-access-cxqrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:29.190973 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.190897    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:29.191012 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.190970    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:29.191012 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.190991    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:29.191880 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.191858    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:29.192118 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.192100    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:29.195171 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.195146    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:29.202279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.202242    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config" pod "0a9adc37-ec78-4112-93c6-e8e02e34bfd2" (UID: "0a9adc37-ec78-4112-93c6-e8e02e34bfd2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:29.206859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206828    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7" exitCode=0
Oct 30 21:15:29.206859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206853    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f" exitCode=0
Oct 30 21:15:29.206859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206860    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e" exitCode=0
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206868    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620" exitCode=0
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206875    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d" exitCode=0
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206880    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798" exitCode=0
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206911    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206907    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"}
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.206995    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"}
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.207040    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"}
Oct 30 21:15:29.207067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.207060    5817 scope.go:122] "RemoveContainer" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.207334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.207083    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"}
Oct 30 21:15:29.207334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.207126    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"}
Oct 30 21:15:29.207334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.207168    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"}
Oct 30 21:15:29.207334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.207207    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0a9adc37-ec78-4112-93c6-e8e02e34bfd2","Type":"ContainerDied","Data":"44ac13a354faa5b5114426cab881bdc6832b975ac3f1ee6c131bc30392e11b6a"}
Oct 30 21:15:29.215599 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.215579    5817 scope.go:122] "RemoveContainer" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.223435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.223411    5817 scope.go:122] "RemoveContainer" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.231230 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.231210    5817 scope.go:122] "RemoveContainer" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.238481 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.238460    5817 scope.go:122] "RemoveContainer" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.246554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.246534    5817 scope.go:122] "RemoveContainer" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.254282 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.254259    5817 scope.go:122] "RemoveContainer" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.262089 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.262067    5817 scope.go:122] "RemoveContainer" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.262430 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:29.262407    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": container with ID starting with b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7 not found: ID does not exist" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.262489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.262442    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"} err="failed to get container status \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": rpc error: code = NotFound desc = could not find container \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": container with ID starting with b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7 not found: ID does not exist"
Oct 30 21:15:29.262489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.262481    5817 scope.go:122] "RemoveContainer" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.262789 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:29.262764    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": container with ID starting with d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f not found: ID does not exist" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.262846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.262797    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"} err="failed to get container status \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": rpc error: code = NotFound desc = could not find container \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": container with ID starting with d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f not found: ID does not exist"
Oct 30 21:15:29.262846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.262813    5817 scope.go:122] "RemoveContainer" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.263069 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:29.263052    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": container with ID starting with b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e not found: ID does not exist" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.263117 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263075    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"} err="failed to get container status \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": rpc error: code = NotFound desc = could not find container \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": container with ID starting with b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e not found: ID does not exist"
Oct 30 21:15:29.263117 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263090    5817 scope.go:122] "RemoveContainer" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.263329 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:29.263311    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": container with ID starting with d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620 not found: ID does not exist" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.263396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263335    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"} err="failed to get container status \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": rpc error: code = NotFound desc = could not find container \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": container with ID starting with d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620 not found: ID does not exist"
Oct 30 21:15:29.263396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263368    5817 scope.go:122] "RemoveContainer" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.263617 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:29.263600    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": container with ID starting with f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d not found: ID does not exist" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.263668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263619    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"} err="failed to get container status \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": rpc error: code = NotFound desc = could not find container \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": container with ID starting with f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d not found: ID does not exist"
Oct 30 21:15:29.263668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263632    5817 scope.go:122] "RemoveContainer" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.263860 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:29.263844    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": container with ID starting with d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798 not found: ID does not exist" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.263901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263863    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"} err="failed to get container status \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": rpc error: code = NotFound desc = could not find container \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": container with ID starting with d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798 not found: ID does not exist"
Oct 30 21:15:29.263901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.263875    5817 scope.go:122] "RemoveContainer" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.264093 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:29.264073    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": container with ID starting with 4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016 not found: ID does not exist" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.264142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264099    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"} err="failed to get container status \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": rpc error: code = NotFound desc = could not find container \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": container with ID starting with 4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016 not found: ID does not exist"
Oct 30 21:15:29.264142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264112    5817 scope.go:122] "RemoveContainer" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.264326 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264308    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"} err="failed to get container status \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": rpc error: code = NotFound desc = could not find container \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": container with ID starting with b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7 not found: ID does not exist"
Oct 30 21:15:29.264397 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264327    5817 scope.go:122] "RemoveContainer" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.264575 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264557    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"} err="failed to get container status \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": rpc error: code = NotFound desc = could not find container \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": container with ID starting with d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f not found: ID does not exist"
Oct 30 21:15:29.264636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264575    5817 scope.go:122] "RemoveContainer" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.264813 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264797    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"} err="failed to get container status \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": rpc error: code = NotFound desc = could not find container \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": container with ID starting with b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e not found: ID does not exist"
Oct 30 21:15:29.264850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.264813    5817 scope.go:122] "RemoveContainer" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.265024 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265002    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"} err="failed to get container status \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": rpc error: code = NotFound desc = could not find container \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": container with ID starting with d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620 not found: ID does not exist"
Oct 30 21:15:29.265024 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265021    5817 scope.go:122] "RemoveContainer" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.265225 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265209    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"} err="failed to get container status \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": rpc error: code = NotFound desc = could not find container \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": container with ID starting with f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d not found: ID does not exist"
Oct 30 21:15:29.265225 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265225    5817 scope.go:122] "RemoveContainer" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.265433 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265417    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"} err="failed to get container status \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": rpc error: code = NotFound desc = could not find container \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": container with ID starting with d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798 not found: ID does not exist"
Oct 30 21:15:29.265433 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265433    5817 scope.go:122] "RemoveContainer" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.265634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265616    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"} err="failed to get container status \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": rpc error: code = NotFound desc = could not find container \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": container with ID starting with 4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016 not found: ID does not exist"
Oct 30 21:15:29.265679 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265633    5817 scope.go:122] "RemoveContainer" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.265828 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265811    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"} err="failed to get container status \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": rpc error: code = NotFound desc = could not find container \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": container with ID starting with b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7 not found: ID does not exist"
Oct 30 21:15:29.265870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.265827    5817 scope.go:122] "RemoveContainer" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.266028 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266012    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"} err="failed to get container status \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": rpc error: code = NotFound desc = could not find container \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": container with ID starting with d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f not found: ID does not exist"
Oct 30 21:15:29.266070 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266028    5817 scope.go:122] "RemoveContainer" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.266256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266238    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"} err="failed to get container status \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": rpc error: code = NotFound desc = could not find container \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": container with ID starting with b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e not found: ID does not exist"
Oct 30 21:15:29.266256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266255    5817 scope.go:122] "RemoveContainer" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.266480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266462    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"} err="failed to get container status \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": rpc error: code = NotFound desc = could not find container \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": container with ID starting with d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620 not found: ID does not exist"
Oct 30 21:15:29.266480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266481    5817 scope.go:122] "RemoveContainer" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.266955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266938    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"} err="failed to get container status \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": rpc error: code = NotFound desc = could not find container \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": container with ID starting with f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d not found: ID does not exist"
Oct 30 21:15:29.266955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.266954    5817 scope.go:122] "RemoveContainer" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.267168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267153    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"} err="failed to get container status \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": rpc error: code = NotFound desc = could not find container \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": container with ID starting with d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798 not found: ID does not exist"
Oct 30 21:15:29.267218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267171    5817 scope.go:122] "RemoveContainer" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.267398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267377    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"} err="failed to get container status \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": rpc error: code = NotFound desc = could not find container \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": container with ID starting with 4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016 not found: ID does not exist"
Oct 30 21:15:29.267442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267398    5817 scope.go:122] "RemoveContainer" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.267619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267602    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"} err="failed to get container status \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": rpc error: code = NotFound desc = could not find container \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": container with ID starting with b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7 not found: ID does not exist"
Oct 30 21:15:29.267659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267619    5817 scope.go:122] "RemoveContainer" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.267826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267810    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"} err="failed to get container status \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": rpc error: code = NotFound desc = could not find container \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": container with ID starting with d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f not found: ID does not exist"
Oct 30 21:15:29.267826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.267824    5817 scope.go:122] "RemoveContainer" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.268034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268017    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"} err="failed to get container status \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": rpc error: code = NotFound desc = could not find container \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": container with ID starting with b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e not found: ID does not exist"
Oct 30 21:15:29.268034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268034    5817 scope.go:122] "RemoveContainer" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.268240 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268224    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"} err="failed to get container status \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": rpc error: code = NotFound desc = could not find container \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": container with ID starting with d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620 not found: ID does not exist"
Oct 30 21:15:29.268240 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268240    5817 scope.go:122] "RemoveContainer" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.268484 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268464    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"} err="failed to get container status \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": rpc error: code = NotFound desc = could not find container \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": container with ID starting with f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d not found: ID does not exist"
Oct 30 21:15:29.268484 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268484    5817 scope.go:122] "RemoveContainer" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.268699 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268682    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"} err="failed to get container status \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": rpc error: code = NotFound desc = could not find container \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": container with ID starting with d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798 not found: ID does not exist"
Oct 30 21:15:29.268699 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268698    5817 scope.go:122] "RemoveContainer" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.268920 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268902    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"} err="failed to get container status \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": rpc error: code = NotFound desc = could not find container \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": container with ID starting with 4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016 not found: ID does not exist"
Oct 30 21:15:29.268963 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.268920    5817 scope.go:122] "RemoveContainer" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.269143 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269125    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"} err="failed to get container status \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": rpc error: code = NotFound desc = could not find container \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": container with ID starting with b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7 not found: ID does not exist"
Oct 30 21:15:29.269185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269144    5817 scope.go:122] "RemoveContainer" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.269387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269369    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"} err="failed to get container status \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": rpc error: code = NotFound desc = could not find container \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": container with ID starting with d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f not found: ID does not exist"
Oct 30 21:15:29.269433 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269388    5817 scope.go:122] "RemoveContainer" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.269607 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269590    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"} err="failed to get container status \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": rpc error: code = NotFound desc = could not find container \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": container with ID starting with b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e not found: ID does not exist"
Oct 30 21:15:29.269654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269606    5817 scope.go:122] "RemoveContainer" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.269827 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269811    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"} err="failed to get container status \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": rpc error: code = NotFound desc = could not find container \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": container with ID starting with d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620 not found: ID does not exist"
Oct 30 21:15:29.269865 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269826    5817 scope.go:122] "RemoveContainer" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.270015 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.269999    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"} err="failed to get container status \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": rpc error: code = NotFound desc = could not find container \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": container with ID starting with f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d not found: ID does not exist"
Oct 30 21:15:29.270051 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270015    5817 scope.go:122] "RemoveContainer" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.270205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270189    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"} err="failed to get container status \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": rpc error: code = NotFound desc = could not find container \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": container with ID starting with d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798 not found: ID does not exist"
Oct 30 21:15:29.270205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270205    5817 scope.go:122] "RemoveContainer" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.270406 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270387    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"} err="failed to get container status \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": rpc error: code = NotFound desc = could not find container \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": container with ID starting with 4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016 not found: ID does not exist"
Oct 30 21:15:29.270450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270407    5817 scope.go:122] "RemoveContainer" containerID="b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"
Oct 30 21:15:29.270616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270599    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7"} err="failed to get container status \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": rpc error: code = NotFound desc = could not find container \"b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7\": container with ID starting with b69ce9ef9c445b9ce5cae14416e4b69f85fd117053d06c5d37c691a0e41581d7 not found: ID does not exist"
Oct 30 21:15:29.270616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270615    5817 scope.go:122] "RemoveContainer" containerID="d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"
Oct 30 21:15:29.270812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270794    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f"} err="failed to get container status \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": rpc error: code = NotFound desc = could not find container \"d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f\": container with ID starting with d0b917c15737153dad594d047a9e999432dc0af320a6aacf8b1d37868c6d380f not found: ID does not exist"
Oct 30 21:15:29.270812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.270811    5817 scope.go:122] "RemoveContainer" containerID="b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"
Oct 30 21:15:29.271023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271008    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e"} err="failed to get container status \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": rpc error: code = NotFound desc = could not find container \"b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e\": container with ID starting with b665831712b94e84dc26c39e54678806bd6040c4b9f51fcfd7405dd0dee9a72e not found: ID does not exist"
Oct 30 21:15:29.271023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271023    5817 scope.go:122] "RemoveContainer" containerID="d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"
Oct 30 21:15:29.271209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271191    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620"} err="failed to get container status \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": rpc error: code = NotFound desc = could not find container \"d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620\": container with ID starting with d243a1fb6fd0bfb469f41e3179d262034d55879bc031cef573419651f6a05620 not found: ID does not exist"
Oct 30 21:15:29.271209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271209    5817 scope.go:122] "RemoveContainer" containerID="f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"
Oct 30 21:15:29.271433 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271418    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d"} err="failed to get container status \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": rpc error: code = NotFound desc = could not find container \"f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d\": container with ID starting with f87800eb359ece57fe4c935629fc98a5334b5fc5f21fcc2a88479459c8eba64d not found: ID does not exist"
Oct 30 21:15:29.271433 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271433    5817 scope.go:122] "RemoveContainer" containerID="d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"
Oct 30 21:15:29.271633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271615    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798"} err="failed to get container status \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": rpc error: code = NotFound desc = could not find container \"d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798\": container with ID starting with d0948893573312d5b2f5b556c00e19ae4e3daa0b79d4c69af49d203bd98a9798 not found: ID does not exist"
Oct 30 21:15:29.271672 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271633    5817 scope.go:122] "RemoveContainer" containerID="4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"
Oct 30 21:15:29.271816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.271800    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016"} err="failed to get container status \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": rpc error: code = NotFound desc = could not find container \"4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016\": container with ID starting with 4fa79fbcc8e0e644d13fb1db65e0fa1542ba39ca8450d809f3cd21d3be94d016 not found: ID does not exist"
Oct 30 21:15:29.288409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288377    5817 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-main-db\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288411    5817 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288430    5817 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-web-config\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288447    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-metrics-client-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288460    5817 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288473    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxqrw\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-kube-api-access-cxqrw\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288486    5817 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288501    5817 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288513    5817 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-config-out\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288525    5817 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-tls-assets\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288539    5817 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288553    5817 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-secret-alertmanager-main-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.288635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.288566    5817 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0a9adc37-ec78-4112-93c6-e8e02e34bfd2-cluster-tls-config\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:29.395968 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.395927    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:29.402311 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.402281    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:29.436154 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436075    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:29.436395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436382    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-web"
Oct 30 21:15:29.436454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436397    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-web"
Oct 30 21:15:29.436454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436414    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-metric"
Oct 30 21:15:29.436454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436420    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-metric"
Oct 30 21:15:29.436454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436427    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy"
Oct 30 21:15:29.436454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436433    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy"
Oct 30 21:15:29.436454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436443    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="config-reloader"
Oct 30 21:15:29.436454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436447    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="config-reloader"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436463    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="alertmanager"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436468    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="alertmanager"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436481    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="prom-label-proxy"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436485    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="prom-label-proxy"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436492    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="init-config-reloader"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436498    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="init-config-reloader"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436547    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="alertmanager"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436556    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="prom-label-proxy"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436562    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="init-config-reloader"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436569    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-web"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436575    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy-metric"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436580    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="kube-rbac-proxy"
Oct 30 21:15:29.436646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.436586    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" containerName="config-reloader"
Oct 30 21:15:29.439255 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.439232    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489547    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1acc732-57dc-4f89-9880-7680cb0d9471-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489591    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1acc732-57dc-4f89-9880-7680cb0d9471-config-out\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489679    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1acc732-57dc-4f89-9880-7680cb0d9471-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489750    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489776    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1acc732-57dc-4f89-9880-7680cb0d9471-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489900 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489847    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1acc732-57dc-4f89-9880-7680cb0d9471-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489930 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489911    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.489999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.489956    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.490042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490022    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.490042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490047    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.490136 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490115    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.490188 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490173    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-web-config\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.490242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490208    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:15:29.490242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490223    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvpck\" (UniqueName: \"kubernetes.io/projected/b1acc732-57dc-4f89-9880-7680cb0d9471-kube-api-access-jvpck\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.490395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490369    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:15:29.490648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490633    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:15:29.490905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490681    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:15:29.490905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490713    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:15:29.490905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490746    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:15:29.490905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490773    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:15:29.490905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490799    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h6tkb\""
Oct 30 21:15:29.491131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.490914    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:15:29.491455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.491437    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:15:29.537379 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.537304    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:29.591098 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591060    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1acc732-57dc-4f89-9880-7680cb0d9471-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591098 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591098    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1acc732-57dc-4f89-9880-7680cb0d9471-config-out\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591118    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1acc732-57dc-4f89-9880-7680cb0d9471-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591138    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591155    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1acc732-57dc-4f89-9880-7680cb0d9471-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591214    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1acc732-57dc-4f89-9880-7680cb0d9471-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591256    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591313    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591367    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591397    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591433    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591495    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-web-config\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591521    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvpck\" (UniqueName: \"kubernetes.io/projected/b1acc732-57dc-4f89-9880-7680cb0d9471-kube-api-access-jvpck\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.591888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591617    5817 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:15:29.591888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591648    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b1acc732-57dc-4f89-9880-7680cb0d9471-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.592193 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.591958    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1acc732-57dc-4f89-9880-7680cb0d9471-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.592425 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.592397    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1acc732-57dc-4f89-9880-7680cb0d9471-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.594654    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b1acc732-57dc-4f89-9880-7680cb0d9471-config-out\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.594682    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-web-config\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.594745    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.594768    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.594773    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-config-volume\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.594857    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b1acc732-57dc-4f89-9880-7680cb0d9471-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.595013    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595381 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.595367    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.595959 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.595941    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b1acc732-57dc-4f89-9880-7680cb0d9471-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.650001 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.649969    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvpck\" (UniqueName: \"kubernetes.io/projected/b1acc732-57dc-4f89-9880-7680cb0d9471-kube-api-access-jvpck\") pod \"alertmanager-main-0\" (UID: \"b1acc732-57dc-4f89-9880-7680cb0d9471\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:29.758526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:29.758488    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:15:30.188847 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:30.188821    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:15:30.191276 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:15:30.191236    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1acc732_57dc_4f89_9880_7680cb0d9471.slice/crio-b1e73104941f16aa8f239a191af06c25827752b438b5db86e3512307fb73f37c WatchSource:0}: Error finding container b1e73104941f16aa8f239a191af06c25827752b438b5db86e3512307fb73f37c: Status 404 returned error can't find the container with id b1e73104941f16aa8f239a191af06c25827752b438b5db86e3512307fb73f37c
Oct 30 21:15:30.219278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:30.219242    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerStarted","Data":"b1e73104941f16aa8f239a191af06c25827752b438b5db86e3512307fb73f37c"}
Oct 30 21:15:30.841437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:30.841402    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a9adc37-ec78-4112-93c6-e8e02e34bfd2" path="/var/lib/kubelet/pods/0a9adc37-ec78-4112-93c6-e8e02e34bfd2/volumes"
Oct 30 21:15:31.224873 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:31.224835    5817 generic.go:356] "Generic (PLEG): container finished" podID="b1acc732-57dc-4f89-9880-7680cb0d9471" containerID="c28ca947e9a653df1a8092072c149a6a7ce9b5f6c3a55c76ebd346216a63fbbc" exitCode=0
Oct 30 21:15:31.225260 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:31.224902    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerDied","Data":"c28ca947e9a653df1a8092072c149a6a7ce9b5f6c3a55c76ebd346216a63fbbc"}
Oct 30 21:15:32.235378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.235326    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerStarted","Data":"bb3161bfbd60e0220833f7c5f55f2e0094c1aa496ceb2175c4805ca9f645331c"}
Oct 30 21:15:32.235772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.235410    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerStarted","Data":"92d4ec0bcafd05f5bd4c874d9a62cf304e3d39eecbccd1c087e06d1f3dce7361"}
Oct 30 21:15:32.235772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.235436    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerStarted","Data":"96d136c43358eb6611784f31d1c9fb73e672d5390c389756afae0602d172ee9e"}
Oct 30 21:15:32.235772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.235461    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerStarted","Data":"0aaf0ce5968f159e12b7b3fc39eb56ad101994d6a4a29803be36d88c605cc1c5"}
Oct 30 21:15:32.235772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.235498    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerStarted","Data":"45d1970d4724ea51fe5b09f0cb1b5075f215159ef8f77a63e45a9d965b4f0eaf"}
Oct 30 21:15:32.235772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.235525    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b1acc732-57dc-4f89-9880-7680cb0d9471","Type":"ContainerStarted","Data":"6ad39cfdd761f21d240bcba4e31a409c438ef510f6a11626fdc8c7830cfed1f4"}
Oct 30 21:15:32.360786 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.360728    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.348100542 podStartE2EDuration="3.360713074s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.612532ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:32.360029299 +0000 UTC m=+67.647751845" watchObservedRunningTime="2026-10-30 21:15:32.360713074 +0000 UTC m=+67.648435611"
Oct 30 21:15:32.671533 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.671448    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-567d7b57f6-qctkw"]
Oct 30 21:15:32.673737 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.673718    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720302 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720260    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-secret-telemeter-client\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720302 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720309    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c4tj\" (UniqueName: \"kubernetes.io/projected/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-kube-api-access-8c4tj\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720541 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720400    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-serving-certs-ca-bundle\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720541 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720420    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-metrics-client-ca\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720541 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720441    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-federate-client-tls\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720541 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720469    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720541 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720503    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.720739 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.720620    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-telemeter-client-tls\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.723746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.723718    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:15:32.723893 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.723833    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-m2rsd\""
Oct 30 21:15:32.723893 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.723833    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:15:32.723893 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.723859    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:15:32.723893 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.723873    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:15:32.723893 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.723861    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:15:32.817951 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.817906    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:15:32.819086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.819058    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-567d7b57f6-qctkw"]
Oct 30 21:15:32.821777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.821745    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c4tj\" (UniqueName: \"kubernetes.io/projected/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-kube-api-access-8c4tj\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.821915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.821791    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-serving-certs-ca-bundle\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.821915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.821807    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-metrics-client-ca\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.821915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.821828    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-federate-client-tls\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.821915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.821887    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.822100 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.821943    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.822100 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.821995    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-telemeter-client-tls\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.822100 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.822076    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-secret-telemeter-client\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.822692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.822660    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-metrics-client-ca\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.822803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.822766    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-serving-certs-ca-bundle\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.823255 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.823231    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.824494 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.824470    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-federate-client-tls\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.824703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.824683    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.824774 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.824682    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-secret-telemeter-client\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.824774 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.824724    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-telemeter-client-tls\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.879940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.879904    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c4tj\" (UniqueName: \"kubernetes.io/projected/3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2-kube-api-access-8c4tj\") pod \"telemeter-client-567d7b57f6-qctkw\" (UID: \"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2\") " pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:32.988930 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:32.988886    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw"
Oct 30 21:15:33.410010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:33.409983    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-567d7b57f6-qctkw"]
Oct 30 21:15:33.412245 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:15:33.412218    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe93ae4_ea86_46a8_8d23_e85c5dc3f2e2.slice/crio-631915e9c78a6607a56d9a9a94eac52cceef54688ee72c595205e4d4e29a83f9 WatchSource:0}: Error finding container 631915e9c78a6607a56d9a9a94eac52cceef54688ee72c595205e4d4e29a83f9: Status 404 returned error can't find the container with id 631915e9c78a6607a56d9a9a94eac52cceef54688ee72c595205e4d4e29a83f9
Oct 30 21:15:34.245851 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.245808    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw" event={"ID":"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2","Type":"ContainerStarted","Data":"631915e9c78a6607a56d9a9a94eac52cceef54688ee72c595205e4d4e29a83f9"}
Oct 30 21:15:34.571758 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.571664    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:15:34.572218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.572058    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="prometheus" containerID="cri-o://584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c" gracePeriod=600
Oct 30 21:15:34.572495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.572448    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-thanos" containerID="cri-o://48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30" gracePeriod=600
Oct 30 21:15:34.572605 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.572559    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy" containerID="cri-o://0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c" gracePeriod=600
Oct 30 21:15:34.572659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.572616    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-web" containerID="cri-o://8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023" gracePeriod=600
Oct 30 21:15:34.572697 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.572667    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="thanos-sidecar" containerID="cri-o://cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0" gracePeriod=600
Oct 30 21:15:34.572727 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.572716    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="config-reloader" containerID="cri-o://e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0" gracePeriod=600
Oct 30 21:15:34.862446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.862411    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:34.946768 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.946733    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.946958 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.946787    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.946958 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.946815    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.946958 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.946841    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947238 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947212    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947291    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947211    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:34.947383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947328    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947436    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947485    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947519    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947564    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947615    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947647    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947685    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947810 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947714    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947810 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947748    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947810 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947785    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947828    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947876    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.947950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.947935    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\") pod \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\" (UID: \"61b3c88c-4121-4a5b-acb2-8d7e78a71243\") "
Oct 30 21:15:34.948214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.948195    5817 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:34.948579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.948549    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:34.949095 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.949067    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:34.949684 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.949656    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:34.950420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950376    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:34.950420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950404    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:34.950549 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950426    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:34.950549 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950438    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.950656 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950625    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.950743 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950707    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "kube-api-access-8zrlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:34.951003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950977    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:34.951083 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.950985    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.952923 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.952894    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.953050 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.953015    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:34.953050 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.953037    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.953647 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.953619    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:34.954304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.954276    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.954587 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.954564    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.954750 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.954717    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:34.966411 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:34.966374    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config" pod "61b3c88c-4121-4a5b-acb2-8d7e78a71243" (UID: "61b3c88c-4121-4a5b-acb2-8d7e78a71243"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:35.049217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049181    5817 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-grpc-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049222    5817 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-metrics-client-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049240    5817 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049257    5817 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049272    5817 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-tls-assets\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049287    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zrlw\" (UniqueName: \"kubernetes.io/projected/61b3c88c-4121-4a5b-acb2-8d7e78a71243-kube-api-access-8zrlw\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049302    5817 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049317    5817 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049332    5817 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-metrics-client-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049367    5817 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049383    5817 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049397    5817 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049407    5817 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-config-out\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049415    5817 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-db\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049425    5817 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049434    5817 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049443    5817 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-secret-kube-rbac-proxy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049453    5817 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/61b3c88c-4121-4a5b-acb2-8d7e78a71243-web-config\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.049519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.049463    5817 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/61b3c88c-4121-4a5b-acb2-8d7e78a71243-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:35.253988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.253949    5817 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30" exitCode=0
Oct 30 21:15:35.253988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.253982    5817 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c" exitCode=0
Oct 30 21:15:35.253988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.253989    5817 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023" exitCode=0
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.253999    5817 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0" exitCode=0
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254009    5817 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0" exitCode=0
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254015    5817 generic.go:356] "Generic (PLEG): container finished" podID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c" exitCode=0
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254047    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"}
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254075    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254133    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"}
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254172    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"}
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254218    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"}
Oct 30 21:15:35.254278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254261    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"}
Oct 30 21:15:35.254700 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254298    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"}
Oct 30 21:15:35.254700 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254339    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"61b3c88c-4121-4a5b-acb2-8d7e78a71243","Type":"ContainerDied","Data":"64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6"}
Oct 30 21:15:35.254700 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.254405    5817 scope.go:122] "RemoveContainer" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.264000 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.263979    5817 scope.go:122] "RemoveContainer" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.282058 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.282020    5817 scope.go:122] "RemoveContainer" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.294328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.294303    5817 scope.go:122] "RemoveContainer" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.295124 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.295090    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b3c88c_4121_4a5b_acb2_8d7e78a71243.slice/crio-64acd159955eee1391b1abf7217d46201e3f737428ec9e1bc746d8e33ee285e6\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b3c88c_4121_4a5b_acb2_8d7e78a71243.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 21:15:35.303525 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.303502    5817 scope.go:122] "RemoveContainer" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.314475 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.314443    5817 scope.go:122] "RemoveContainer" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.323529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.323494    5817 scope.go:122] "RemoveContainer" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.332305 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.332277    5817 scope.go:122] "RemoveContainer" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.332723 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.332693    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": container with ID starting with 48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30 not found: ID does not exist" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.332809 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.332737    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"} err="failed to get container status \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": rpc error: code = NotFound desc = could not find container \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": container with ID starting with 48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30 not found: ID does not exist"
Oct 30 21:15:35.332809 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.332760    5817 scope.go:122] "RemoveContainer" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.333108 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.333085    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": container with ID starting with 0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c not found: ID does not exist" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.333329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.333118    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"} err="failed to get container status \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": rpc error: code = NotFound desc = could not find container \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": container with ID starting with 0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c not found: ID does not exist"
Oct 30 21:15:35.333329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.333145    5817 scope.go:122] "RemoveContainer" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.333543 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.333510    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": container with ID starting with 8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023 not found: ID does not exist" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.333592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.333554    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"} err="failed to get container status \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": rpc error: code = NotFound desc = could not find container \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": container with ID starting with 8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023 not found: ID does not exist"
Oct 30 21:15:35.333592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.333569    5817 scope.go:122] "RemoveContainer" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.333842 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.333812    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": container with ID starting with cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0 not found: ID does not exist" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.333883 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.333849    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"} err="failed to get container status \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": rpc error: code = NotFound desc = could not find container \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": container with ID starting with cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0 not found: ID does not exist"
Oct 30 21:15:35.333883 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.333869    5817 scope.go:122] "RemoveContainer" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.334185 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.334161    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": container with ID starting with e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0 not found: ID does not exist" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.334234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.334191    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"} err="failed to get container status \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": rpc error: code = NotFound desc = could not find container \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": container with ID starting with e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0 not found: ID does not exist"
Oct 30 21:15:35.334234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.334209    5817 scope.go:122] "RemoveContainer" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.334484 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.334467    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": container with ID starting with 584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c not found: ID does not exist" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.334534 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.334490    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"} err="failed to get container status \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": rpc error: code = NotFound desc = could not find container \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": container with ID starting with 584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c not found: ID does not exist"
Oct 30 21:15:35.334534 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.334504    5817 scope.go:122] "RemoveContainer" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.334793 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:35.334771    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": container with ID starting with 3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244 not found: ID does not exist" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.334857 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.334798    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"} err="failed to get container status \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": rpc error: code = NotFound desc = could not find container \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": container with ID starting with 3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244 not found: ID does not exist"
Oct 30 21:15:35.334857 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.334819    5817 scope.go:122] "RemoveContainer" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.335086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335063    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"} err="failed to get container status \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": rpc error: code = NotFound desc = could not find container \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": container with ID starting with 48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30 not found: ID does not exist"
Oct 30 21:15:35.335086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335084    5817 scope.go:122] "RemoveContainer" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.335338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335318    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"} err="failed to get container status \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": rpc error: code = NotFound desc = could not find container \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": container with ID starting with 0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c not found: ID does not exist"
Oct 30 21:15:35.335414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335338    5817 scope.go:122] "RemoveContainer" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.335685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335659    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"} err="failed to get container status \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": rpc error: code = NotFound desc = could not find container \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": container with ID starting with 8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023 not found: ID does not exist"
Oct 30 21:15:35.335685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335685    5817 scope.go:122] "RemoveContainer" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.335977 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335958    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"} err="failed to get container status \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": rpc error: code = NotFound desc = could not find container \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": container with ID starting with cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0 not found: ID does not exist"
Oct 30 21:15:35.336038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.335978    5817 scope.go:122] "RemoveContainer" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.336210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.336197    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"} err="failed to get container status \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": rpc error: code = NotFound desc = could not find container \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": container with ID starting with e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0 not found: ID does not exist"
Oct 30 21:15:35.336277 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.336211    5817 scope.go:122] "RemoveContainer" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.336490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.336471    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"} err="failed to get container status \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": rpc error: code = NotFound desc = could not find container \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": container with ID starting with 584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c not found: ID does not exist"
Oct 30 21:15:35.336490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.336490    5817 scope.go:122] "RemoveContainer" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.336777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.336753    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"} err="failed to get container status \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": rpc error: code = NotFound desc = could not find container \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": container with ID starting with 3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244 not found: ID does not exist"
Oct 30 21:15:35.336860 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.336779    5817 scope.go:122] "RemoveContainer" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.337035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337013    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"} err="failed to get container status \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": rpc error: code = NotFound desc = could not find container \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": container with ID starting with 48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30 not found: ID does not exist"
Oct 30 21:15:35.337081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337036    5817 scope.go:122] "RemoveContainer" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.337334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337314    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"} err="failed to get container status \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": rpc error: code = NotFound desc = could not find container \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": container with ID starting with 0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c not found: ID does not exist"
Oct 30 21:15:35.337398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337334    5817 scope.go:122] "RemoveContainer" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.337596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337576    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"} err="failed to get container status \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": rpc error: code = NotFound desc = could not find container \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": container with ID starting with 8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023 not found: ID does not exist"
Oct 30 21:15:35.337651 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337597    5817 scope.go:122] "RemoveContainer" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.337892 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337866    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"} err="failed to get container status \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": rpc error: code = NotFound desc = could not find container \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": container with ID starting with cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0 not found: ID does not exist"
Oct 30 21:15:35.337958 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.337893    5817 scope.go:122] "RemoveContainer" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.338152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338133    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"} err="failed to get container status \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": rpc error: code = NotFound desc = could not find container \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": container with ID starting with e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0 not found: ID does not exist"
Oct 30 21:15:35.338152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338152    5817 scope.go:122] "RemoveContainer" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.338439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338412    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"} err="failed to get container status \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": rpc error: code = NotFound desc = could not find container \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": container with ID starting with 584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c not found: ID does not exist"
Oct 30 21:15:35.338439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338438    5817 scope.go:122] "RemoveContainer" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.338674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338654    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"} err="failed to get container status \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": rpc error: code = NotFound desc = could not find container \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": container with ID starting with 3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244 not found: ID does not exist"
Oct 30 21:15:35.338724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338674    5817 scope.go:122] "RemoveContainer" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.338896 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338876    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"} err="failed to get container status \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": rpc error: code = NotFound desc = could not find container \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": container with ID starting with 48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30 not found: ID does not exist"
Oct 30 21:15:35.338953 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.338897    5817 scope.go:122] "RemoveContainer" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.339141 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339120    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"} err="failed to get container status \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": rpc error: code = NotFound desc = could not find container \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": container with ID starting with 0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c not found: ID does not exist"
Oct 30 21:15:35.339203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339142    5817 scope.go:122] "RemoveContainer" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.339399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339378    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"} err="failed to get container status \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": rpc error: code = NotFound desc = could not find container \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": container with ID starting with 8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023 not found: ID does not exist"
Oct 30 21:15:35.339469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339399    5817 scope.go:122] "RemoveContainer" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.339671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339642    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"} err="failed to get container status \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": rpc error: code = NotFound desc = could not find container \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": container with ID starting with cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0 not found: ID does not exist"
Oct 30 21:15:35.339671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339671    5817 scope.go:122] "RemoveContainer" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.339915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339896    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"} err="failed to get container status \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": rpc error: code = NotFound desc = could not find container \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": container with ID starting with e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0 not found: ID does not exist"
Oct 30 21:15:35.339989 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.339916    5817 scope.go:122] "RemoveContainer" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.340196 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.340171    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"} err="failed to get container status \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": rpc error: code = NotFound desc = could not find container \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": container with ID starting with 584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c not found: ID does not exist"
Oct 30 21:15:35.340246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.340197    5817 scope.go:122] "RemoveContainer" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.340590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.340559    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"} err="failed to get container status \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": rpc error: code = NotFound desc = could not find container \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": container with ID starting with 3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244 not found: ID does not exist"
Oct 30 21:15:35.340590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.340588    5817 scope.go:122] "RemoveContainer" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.340822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.340805    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"} err="failed to get container status \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": rpc error: code = NotFound desc = could not find container \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": container with ID starting with 48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30 not found: ID does not exist"
Oct 30 21:15:35.340888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.340823    5817 scope.go:122] "RemoveContainer" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.341081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341058    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"} err="failed to get container status \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": rpc error: code = NotFound desc = could not find container \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": container with ID starting with 0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c not found: ID does not exist"
Oct 30 21:15:35.341148 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341081    5817 scope.go:122] "RemoveContainer" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.341326 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341309    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"} err="failed to get container status \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": rpc error: code = NotFound desc = could not find container \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": container with ID starting with 8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023 not found: ID does not exist"
Oct 30 21:15:35.341403 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341327    5817 scope.go:122] "RemoveContainer" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.341632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341610    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"} err="failed to get container status \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": rpc error: code = NotFound desc = could not find container \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": container with ID starting with cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0 not found: ID does not exist"
Oct 30 21:15:35.341632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341631    5817 scope.go:122] "RemoveContainer" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.341934 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341915    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"} err="failed to get container status \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": rpc error: code = NotFound desc = could not find container \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": container with ID starting with e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0 not found: ID does not exist"
Oct 30 21:15:35.341934 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.341933    5817 scope.go:122] "RemoveContainer" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.342177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.342160    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"} err="failed to get container status \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": rpc error: code = NotFound desc = could not find container \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": container with ID starting with 584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c not found: ID does not exist"
Oct 30 21:15:35.342177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.342176    5817 scope.go:122] "RemoveContainer" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.342481 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.342456    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"} err="failed to get container status \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": rpc error: code = NotFound desc = could not find container \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": container with ID starting with 3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244 not found: ID does not exist"
Oct 30 21:15:35.342596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.342482    5817 scope.go:122] "RemoveContainer" containerID="48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"
Oct 30 21:15:35.342745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.342722    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30"} err="failed to get container status \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": rpc error: code = NotFound desc = could not find container \"48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30\": container with ID starting with 48204f406e0cd84ace9967d9715c09e9c0eebfb0dc544bedabc9e94158919f30 not found: ID does not exist"
Oct 30 21:15:35.342745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.342744    5817 scope.go:122] "RemoveContainer" containerID="0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"
Oct 30 21:15:35.343025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343003    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c"} err="failed to get container status \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": rpc error: code = NotFound desc = could not find container \"0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c\": container with ID starting with 0e054c22af5140097db6ac670e0a09c90f777ff02fe50910f1a2855f9625ea7c not found: ID does not exist"
Oct 30 21:15:35.343116 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343028    5817 scope.go:122] "RemoveContainer" containerID="8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"
Oct 30 21:15:35.343304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343282    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023"} err="failed to get container status \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": rpc error: code = NotFound desc = could not find container \"8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023\": container with ID starting with 8e010b5e3811303243829f42f53636f51eb6ef6e1c4ad0a8943f9303747ca023 not found: ID does not exist"
Oct 30 21:15:35.343389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343305    5817 scope.go:122] "RemoveContainer" containerID="cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"
Oct 30 21:15:35.343639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343611    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0"} err="failed to get container status \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": rpc error: code = NotFound desc = could not find container \"cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0\": container with ID starting with cfbb422e9e6ea7406008d322302df4f705f9f1f86522897a81b48e28169633d0 not found: ID does not exist"
Oct 30 21:15:35.343639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343638    5817 scope.go:122] "RemoveContainer" containerID="e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"
Oct 30 21:15:35.343915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343894    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0"} err="failed to get container status \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": rpc error: code = NotFound desc = could not find container \"e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0\": container with ID starting with e50a9800b63661dd740da1c4a53f93c7d946da4c96692732a4962455c4dc16d0 not found: ID does not exist"
Oct 30 21:15:35.343986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.343916    5817 scope.go:122] "RemoveContainer" containerID="584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"
Oct 30 21:15:35.344180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.344159    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c"} err="failed to get container status \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": rpc error: code = NotFound desc = could not find container \"584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c\": container with ID starting with 584651c5a2d9fb5d0a314ceb6ca56104640f4656c6d5abeff951181c6b768f3c not found: ID does not exist"
Oct 30 21:15:35.344244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.344192    5817 scope.go:122] "RemoveContainer" containerID="3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"
Oct 30 21:15:35.344526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.344495    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244"} err="failed to get container status \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": rpc error: code = NotFound desc = could not find container \"3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244\": container with ID starting with 3051f05912f467fa8e3e5d9072a6040f005132c5aacb1ec05c828762b3f71244 not found: ID does not exist"
Oct 30 21:15:35.414145 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.414109    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:15:35.462335 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462293    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:15:35.462615 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462376    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:15:35.462791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462765    5817 cpu_manager.go:414] "containerMap: removing container" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-web"
Oct 30 21:15:35.462791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462787    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-web"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462812    5817 cpu_manager.go:414] "containerMap: removing container" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="prometheus"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462820    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="prometheus"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462830    5817 cpu_manager.go:414] "containerMap: removing container" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-thanos"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462838    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-thanos"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462852    5817 cpu_manager.go:414] "containerMap: removing container" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="config-reloader"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462859    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="config-reloader"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462876    5817 cpu_manager.go:414] "containerMap: removing container" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="init-config-reloader"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462883    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="init-config-reloader"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462900    5817 cpu_manager.go:414] "containerMap: removing container" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="thanos-sidecar"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462907    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="thanos-sidecar"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462926    5817 cpu_manager.go:414] "containerMap: removing container" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy"
Oct 30 21:15:35.462962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.462934    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy"
Oct 30 21:15:35.463613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.463012    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="thanos-sidecar"
Oct 30 21:15:35.463613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.463025    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy"
Oct 30 21:15:35.463613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.463036    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-web"
Oct 30 21:15:35.463613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.463047    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="prometheus"
Oct 30 21:15:35.463613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.463057    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="kube-rbac-proxy-thanos"
Oct 30 21:15:35.463613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.463066    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" containerName="config-reloader"
Oct 30 21:15:35.467779 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.467750    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518044    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518052    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518155    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518201    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518239    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k9bmv\""
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518273    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518412    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:15:35.518527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518432    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:15:35.519023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518661    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-68m835m929agv\""
Oct 30 21:15:35.519023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518709    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:15:35.519023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.518772    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:15:35.519107 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.519039    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:15:35.519317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.519213    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:15:35.519317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.519256    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:15:35.519317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.519263    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:15:35.555945 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.555901    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.555963    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.555993    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556014    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556053    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e02771be-80a2-4d57-bccf-c401aa52ca2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556078    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556102    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556131    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556174    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e02771be-80a2-4d57-bccf-c401aa52ca2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556192    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556216    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556240    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556275    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkkqb\" (UniqueName: \"kubernetes.io/projected/e02771be-80a2-4d57-bccf-c401aa52ca2a-kube-api-access-wkkqb\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556291    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-config\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556309    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556334    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556390    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556425    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556460    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.556840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.556482    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.568281 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.566803    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:15:35.568281 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.566889    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:15:35.568281 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.568173    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:15:35.657432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657396    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657439    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657459    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657489    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657522    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e02771be-80a2-4d57-bccf-c401aa52ca2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657549    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657590    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657619    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657659    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e02771be-80a2-4d57-bccf-c401aa52ca2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657681    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657705    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657731    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657779    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkkqb\" (UniqueName: \"kubernetes.io/projected/e02771be-80a2-4d57-bccf-c401aa52ca2a-kube-api-access-wkkqb\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657805    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-config\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657831    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657858    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657896    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657926    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657954    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.657988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.657978    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.658875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.658226    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.660580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.660241    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.660580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.660497    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.661115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.661092    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.664414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.661948    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.664414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.662741    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.664414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.663568    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.664414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.664035    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-config\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.665083 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.665055    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.665185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.665101    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.669470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.665711    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.669470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.665741    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.669470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.665835    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.669470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.666001    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e02771be-80a2-4d57-bccf-c401aa52ca2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.669470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.666158    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.669470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.666283    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.669470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.668884    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e02771be-80a2-4d57-bccf-c401aa52ca2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.670880 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.670850    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e02771be-80a2-4d57-bccf-c401aa52ca2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.671510 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.671485    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e02771be-80a2-4d57-bccf-c401aa52ca2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.725139 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.725098    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkkqb\" (UniqueName: \"kubernetes.io/projected/e02771be-80a2-4d57-bccf-c401aa52ca2a-kube-api-access-wkkqb\") pod \"prometheus-k8s-0\" (UID: \"e02771be-80a2-4d57-bccf-c401aa52ca2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:35.793303 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:35.793268    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:36.263707 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:36.263674    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw" event={"ID":"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2","Type":"ContainerStarted","Data":"f686330016166ef1547daceaade8cd8255c4c533869b225178c239331201f075"}
Oct 30 21:15:36.263885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:36.263763    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw" event={"ID":"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2","Type":"ContainerStarted","Data":"f9850a1b849f7528d425eb86107ddab0eb9751e940ce6ad663be6f5cd8e33b41"}
Oct 30 21:15:36.263885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:36.263786    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw" event={"ID":"3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2","Type":"ContainerStarted","Data":"8d044b591a5314ed27ba5e0522677259ae186b9850b9bf880feb8226a2159e0b"}
Oct 30 21:15:36.266897 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:36.266873    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:15:36.269290 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:15:36.269264    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode02771be_80a2_4d57_bccf_c401aa52ca2a.slice/crio-195eef7a97b82c36a427ac49e88baa0d9534bbe731d76ad191cd02ce50a5f99b WatchSource:0}: Error finding container 195eef7a97b82c36a427ac49e88baa0d9534bbe731d76ad191cd02ce50a5f99b: Status 404 returned error can't find the container with id 195eef7a97b82c36a427ac49e88baa0d9534bbe731d76ad191cd02ce50a5f99b
Oct 30 21:15:36.381657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:36.381598    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-567d7b57f6-qctkw" podStartSLOduration=2.049608446 podStartE2EDuration="4.381583527s" totalImagesPullingTime="2.331975081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:36.380483933 +0000 UTC m=+71.668206484" watchObservedRunningTime="2026-10-30 21:15:36.381583527 +0000 UTC m=+71.669306067"
Oct 30 21:15:36.841332 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:36.841298    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61b3c88c-4121-4a5b-acb2-8d7e78a71243" path="/var/lib/kubelet/pods/61b3c88c-4121-4a5b-acb2-8d7e78a71243/volumes"
Oct 30 21:15:37.269149 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:37.269107    5817 generic.go:356] "Generic (PLEG): container finished" podID="e02771be-80a2-4d57-bccf-c401aa52ca2a" containerID="7d67ce1a15b50096510d2f5ea89adda9a8c2ee278e3ec0b13d82df432c0f7954" exitCode=0
Oct 30 21:15:37.269325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:37.269199    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerDied","Data":"7d67ce1a15b50096510d2f5ea89adda9a8c2ee278e3ec0b13d82df432c0f7954"}
Oct 30 21:15:37.269325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:37.269288    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerStarted","Data":"195eef7a97b82c36a427ac49e88baa0d9534bbe731d76ad191cd02ce50a5f99b"}
Oct 30 21:15:38.277699 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:38.277660    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerStarted","Data":"c222e5c83d41786e98209b780db9b15f379bcb62c88542f6c615e0964f9682aa"}
Oct 30 21:15:38.278110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:38.277761    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerStarted","Data":"5d2e3a378386a96ffccaaae1dc081d8568d5df061ddb2602e557a1bb833f7e29"}
Oct 30 21:15:38.278110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:38.277792    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerStarted","Data":"223ae4ac5a171b2b9efbce1450a2dc48819e67bd24193bb45a0c7e64d9daf2e6"}
Oct 30 21:15:38.278110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:38.277817    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerStarted","Data":"57f3ea640bb955b270f692facc064ea3ddd2c0b4fd50428126f03f0bb347f3d8"}
Oct 30 21:15:38.278110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:38.277843    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerStarted","Data":"e40572c264bd46fb10ef79c34f3fa4c6594dc3bc994046a39bc21cbd8c7847fa"}
Oct 30 21:15:38.278110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:38.277866    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e02771be-80a2-4d57-bccf-c401aa52ca2a","Type":"ContainerStarted","Data":"5bb2e7fdff79d31d90ebdb0866d94797029e18ede23b7c77f5a2dd6afc76f89e"}
Oct 30 21:15:38.404491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:38.404431    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.39205159 podStartE2EDuration="3.404417108s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.365518ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:38.40347994 +0000 UTC m=+73.691202486" watchObservedRunningTime="2026-10-30 21:15:38.404417108 +0000 UTC m=+73.692139682"
Oct 30 21:15:40.793637 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:15:40.793592    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:15:46.337199 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:15:46.337152    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:16:16.338501 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:16:16.338411    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:16:35.794252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:16:35.794205    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:16:35.811120 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:16:35.811086    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:16:36.515019 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:16:36.514988    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:16:46.339018 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:16:46.338972    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:17:16.336722 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:17:16.336677    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:17:46.336893 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:17:46.336844    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:18:16.336562 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:18:16.336521    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:18:46.335759 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:18:46.335713    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:19:16.336341 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:19:16.336263    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:19:31.181503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.181471    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"]
Oct 30 21:19:31.183646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.183630    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.235882 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.235859    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:19:31.282787 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.282760    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 21:19:31.282894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.282794    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-vlpvd\""
Oct 30 21:19:31.283761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.283742    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"]
Oct 30 21:19:31.294452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.294432    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5da2d011-dcb0-41bf-a4d0-fe54a211e70b-tmp\") pod \"jobset-operator-674cd59dc5-26brf\" (UID: \"5da2d011-dcb0-41bf-a4d0-fe54a211e70b\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.294508 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.294493    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q56mg\" (UniqueName: \"kubernetes.io/projected/5da2d011-dcb0-41bf-a4d0-fe54a211e70b-kube-api-access-q56mg\") pod \"jobset-operator-674cd59dc5-26brf\" (UID: \"5da2d011-dcb0-41bf-a4d0-fe54a211e70b\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.395133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.395111    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q56mg\" (UniqueName: \"kubernetes.io/projected/5da2d011-dcb0-41bf-a4d0-fe54a211e70b-kube-api-access-q56mg\") pod \"jobset-operator-674cd59dc5-26brf\" (UID: \"5da2d011-dcb0-41bf-a4d0-fe54a211e70b\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.395233 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.395156    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5da2d011-dcb0-41bf-a4d0-fe54a211e70b-tmp\") pod \"jobset-operator-674cd59dc5-26brf\" (UID: \"5da2d011-dcb0-41bf-a4d0-fe54a211e70b\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.395509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.395491    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5da2d011-dcb0-41bf-a4d0-fe54a211e70b-tmp\") pod \"jobset-operator-674cd59dc5-26brf\" (UID: \"5da2d011-dcb0-41bf-a4d0-fe54a211e70b\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.453084 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.453056    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q56mg\" (UniqueName: \"kubernetes.io/projected/5da2d011-dcb0-41bf-a4d0-fe54a211e70b-kube-api-access-q56mg\") pod \"jobset-operator-674cd59dc5-26brf\" (UID: \"5da2d011-dcb0-41bf-a4d0-fe54a211e70b\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.503597 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.503570    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"
Oct 30 21:19:31.868157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.868080    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-26brf"]
Oct 30 21:19:31.871493 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:19:31.871467    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5da2d011_dcb0_41bf_a4d0_fe54a211e70b.slice/crio-4e89c4225a3dfed7575c81295e4f38c080f08a0d7c858f8864a671f7e203db58 WatchSource:0}: Error finding container 4e89c4225a3dfed7575c81295e4f38c080f08a0d7c858f8864a671f7e203db58: Status 404 returned error can't find the container with id 4e89c4225a3dfed7575c81295e4f38c080f08a0d7c858f8864a671f7e203db58
Oct 30 21:19:31.872965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:31.872947    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:19:32.136863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:32.136774    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf" event={"ID":"5da2d011-dcb0-41bf-a4d0-fe54a211e70b","Type":"ContainerStarted","Data":"4e89c4225a3dfed7575c81295e4f38c080f08a0d7c858f8864a671f7e203db58"}
Oct 30 21:19:36.154687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:36.154655    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf" event={"ID":"5da2d011-dcb0-41bf-a4d0-fe54a211e70b","Type":"ContainerStarted","Data":"167239713f2f2504b6c0b06ea99d854ffacd8193d7a1bba0d1d4cb3c91310ed8"}
Oct 30 21:19:36.266397 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:36.266307    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-26brf" podStartSLOduration=1.172129104 podStartE2EDuration="5.266294915s" totalImagesPullingTime="4.094165811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:36.265615468 +0000 UTC m=+311.553338015" watchObservedRunningTime="2026-10-30 21:19:36.266294915 +0000 UTC m=+311.554017460"
Oct 30 21:19:39.265049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.265016    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"]
Oct 30 21:19:39.267229 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.267212    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.364721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.364689    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-metrics-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.364851 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.364730    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/db358ecc-02e1-4f54-88d3-c73c007f23ff-manifests\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.364851 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.364811    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.364851 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.364842    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.364962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.364858    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jggk4\" (UniqueName: \"kubernetes.io/projected/db358ecc-02e1-4f54-88d3-c73c007f23ff-kube-api-access-jggk4\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.364962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.364916    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db358ecc-02e1-4f54-88d3-c73c007f23ff-tmp\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.392315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.392294    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:19:39.392576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.392561    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:19:39.392664 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.392584    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:19:39.392664 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.392623    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:19:39.392664 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.392627    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-5lv4p\""
Oct 30 21:19:39.392754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.392670    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:19:39.465660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.465636    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.465800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.465664    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jggk4\" (UniqueName: \"kubernetes.io/projected/db358ecc-02e1-4f54-88d3-c73c007f23ff-kube-api-access-jggk4\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.465800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.465686    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db358ecc-02e1-4f54-88d3-c73c007f23ff-tmp\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.465800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.465721    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-metrics-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.465800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.465737    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/db358ecc-02e1-4f54-88d3-c73c007f23ff-manifests\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.465800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.465777    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.466313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.466287    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/db358ecc-02e1-4f54-88d3-c73c007f23ff-manifests\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.468472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.468429    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-metrics-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.468570 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.468488    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db358ecc-02e1-4f54-88d3-c73c007f23ff-tmp\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.468613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.468588    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.468740 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.468725    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db358ecc-02e1-4f54-88d3-c73c007f23ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.500247 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.500222    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"]
Oct 30 21:19:39.524587 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.524529    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jggk4\" (UniqueName: \"kubernetes.io/projected/db358ecc-02e1-4f54-88d3-c73c007f23ff-kube-api-access-jggk4\") pod \"opendatahub-operator-controller-manager-695f8457f8-5nk64\" (UID: \"db358ecc-02e1-4f54-88d3-c73c007f23ff\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:39.585116 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:39.585088    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:40.085271 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:40.085235    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"]
Oct 30 21:19:40.088715 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:19:40.088689    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb358ecc_02e1_4f54_88d3_c73c007f23ff.slice/crio-52067112438e90acb4b6b6cef83130ab8101681ab63d288bec9f281c13f2afe8 WatchSource:0}: Error finding container 52067112438e90acb4b6b6cef83130ab8101681ab63d288bec9f281c13f2afe8: Status 404 returned error can't find the container with id 52067112438e90acb4b6b6cef83130ab8101681ab63d288bec9f281c13f2afe8
Oct 30 21:19:40.171552 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:40.171523    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64" event={"ID":"db358ecc-02e1-4f54-88d3-c73c007f23ff","Type":"ContainerStarted","Data":"52067112438e90acb4b6b6cef83130ab8101681ab63d288bec9f281c13f2afe8"}
Oct 30 21:19:44.191438 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:44.191403    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64" event={"ID":"db358ecc-02e1-4f54-88d3-c73c007f23ff","Type":"ContainerStarted","Data":"531e57d6d634364f550276441837268959e1613d0b1a053d73ad9826e47433da"}
Oct 30 21:19:45.196320 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:45.196285    5817 generic.go:356] "Generic (PLEG): container finished" podID="db358ecc-02e1-4f54-88d3-c73c007f23ff" containerID="531e57d6d634364f550276441837268959e1613d0b1a053d73ad9826e47433da" exitCode=0
Oct 30 21:19:45.196752 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:45.196363    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64" event={"ID":"db358ecc-02e1-4f54-88d3-c73c007f23ff","Type":"ContainerDied","Data":"531e57d6d634364f550276441837268959e1613d0b1a053d73ad9826e47433da"}
Oct 30 21:19:46.202816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:46.202783    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64" event={"ID":"db358ecc-02e1-4f54-88d3-c73c007f23ff","Type":"ContainerStarted","Data":"f34b2d908e926f4d8e82ef4b4b3255e11e1ae3e67fc362208b8fb396ff55c7a1"}
Oct 30 21:19:46.203203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:46.202895    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:19:46.335285 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:19:46.335222    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:19:46.374899 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:46.374858    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64" podStartSLOduration=2.458388335 podStartE2EDuration="7.3748473s" totalImagesPullingTime="4.419033805s" totalInitContainerRuntime="497.42516ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:19:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:19:46.37322564 +0000 UTC m=+321.660948187" watchObservedRunningTime="2026-10-30 21:19:46.3748473 +0000 UTC m=+321.662569846"
Oct 30 21:19:57.209213 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:19:57.209183    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-695f8457f8-5nk64"
Oct 30 21:20:05.492805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.492767    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"]
Oct 30 21:20:05.496295 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.496269    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:05.580327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.580298    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrfwn\" (UniqueName: \"kubernetes.io/projected/456ba45c-c157-4be2-81fa-a8dcbffcb604-kube-api-access-vrfwn\") pod \"cert-manager-cainjector-5c57bbd4fc-p97rd\" (UID: \"456ba45c-c157-4be2-81fa-a8dcbffcb604\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:05.580479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.580370    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/456ba45c-c157-4be2-81fa-a8dcbffcb604-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-p97rd\" (UID: \"456ba45c-c157-4be2-81fa-a8dcbffcb604\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:05.592660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.592633    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 21:20:05.592794 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.592664    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qn6jv\""
Oct 30 21:20:05.592794 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.592744    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 21:20:05.593743 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.593725    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"]
Oct 30 21:20:05.681165 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.681139    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrfwn\" (UniqueName: \"kubernetes.io/projected/456ba45c-c157-4be2-81fa-a8dcbffcb604-kube-api-access-vrfwn\") pod \"cert-manager-cainjector-5c57bbd4fc-p97rd\" (UID: \"456ba45c-c157-4be2-81fa-a8dcbffcb604\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:05.681305 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.681185    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/456ba45c-c157-4be2-81fa-a8dcbffcb604-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-p97rd\" (UID: \"456ba45c-c157-4be2-81fa-a8dcbffcb604\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:05.742016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.741994    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/456ba45c-c157-4be2-81fa-a8dcbffcb604-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-p97rd\" (UID: \"456ba45c-c157-4be2-81fa-a8dcbffcb604\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:05.742138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.742065    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrfwn\" (UniqueName: \"kubernetes.io/projected/456ba45c-c157-4be2-81fa-a8dcbffcb604-kube-api-access-vrfwn\") pod \"cert-manager-cainjector-5c57bbd4fc-p97rd\" (UID: \"456ba45c-c157-4be2-81fa-a8dcbffcb604\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:05.812243 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:05.812184    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"
Oct 30 21:20:06.179798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:06.179730    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd"]
Oct 30 21:20:06.183285 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:20:06.183251    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod456ba45c_c157_4be2_81fa_a8dcbffcb604.slice/crio-15b5c3783cfdfcbe50b9fa92f2f29782904feacbcd44f35eb5781f4337619876 WatchSource:0}: Error finding container 15b5c3783cfdfcbe50b9fa92f2f29782904feacbcd44f35eb5781f4337619876: Status 404 returned error can't find the container with id 15b5c3783cfdfcbe50b9fa92f2f29782904feacbcd44f35eb5781f4337619876
Oct 30 21:20:06.285705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:06.285673    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd" event={"ID":"456ba45c-c157-4be2-81fa-a8dcbffcb604","Type":"ContainerStarted","Data":"15b5c3783cfdfcbe50b9fa92f2f29782904feacbcd44f35eb5781f4337619876"}
Oct 30 21:20:09.668362 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.668307    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"]
Oct 30 21:20:09.672159 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.672137    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:09.771996 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.771965    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-wb6gf\""
Oct 30 21:20:09.773167 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.773147    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"]
Oct 30 21:20:09.817830 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.817801    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86024e40-8bdb-4fcd-b5e8-1cb2045a5925-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-htr9j\" (UID: \"86024e40-8bdb-4fcd-b5e8-1cb2045a5925\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:09.817978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.817869    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gg2m\" (UniqueName: \"kubernetes.io/projected/86024e40-8bdb-4fcd-b5e8-1cb2045a5925-kube-api-access-6gg2m\") pod \"cert-manager-webhook-5dbbbf75cc-htr9j\" (UID: \"86024e40-8bdb-4fcd-b5e8-1cb2045a5925\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:09.919083 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.919011    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gg2m\" (UniqueName: \"kubernetes.io/projected/86024e40-8bdb-4fcd-b5e8-1cb2045a5925-kube-api-access-6gg2m\") pod \"cert-manager-webhook-5dbbbf75cc-htr9j\" (UID: \"86024e40-8bdb-4fcd-b5e8-1cb2045a5925\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:09.919261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.919104    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86024e40-8bdb-4fcd-b5e8-1cb2045a5925-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-htr9j\" (UID: \"86024e40-8bdb-4fcd-b5e8-1cb2045a5925\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:09.980087 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.980061    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gg2m\" (UniqueName: \"kubernetes.io/projected/86024e40-8bdb-4fcd-b5e8-1cb2045a5925-kube-api-access-6gg2m\") pod \"cert-manager-webhook-5dbbbf75cc-htr9j\" (UID: \"86024e40-8bdb-4fcd-b5e8-1cb2045a5925\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:09.980639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.980615    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86024e40-8bdb-4fcd-b5e8-1cb2045a5925-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-htr9j\" (UID: \"86024e40-8bdb-4fcd-b5e8-1cb2045a5925\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:09.985495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:09.985473    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:10.071772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.071312    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"]
Oct 30 21:20:10.077927 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.077903    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.130083 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.130052    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:20:10.132633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.132603    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"]
Oct 30 21:20:10.133897 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.133875    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:20:10.134017 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.134002    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:20:10.222499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.222463    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-tls-certs\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.222698 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.222555    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-tmp\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.222698 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.222600    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cptdm\" (UniqueName: \"kubernetes.io/projected/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-kube-api-access-cptdm\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.226717 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.226684    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:20:10.226717 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.226697    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pwtw7\""
Oct 30 21:20:10.323047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.323015    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-tmp\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.323232 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.323068    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cptdm\" (UniqueName: \"kubernetes.io/projected/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-kube-api-access-cptdm\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.323232 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.323157    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-tls-certs\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.325606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.325578    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-tmp\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.326043 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.326022    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-tls-certs\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.364148 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.364079    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"]
Oct 30 21:20:10.366704 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:20:10.366661    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86024e40_8bdb_4fcd_b5e8_1cb2045a5925.slice/crio-5b1eecb817b23df979964d1471d4d2016d182a20b15ecafcd90c100af97b2a41 WatchSource:0}: Error finding container 5b1eecb817b23df979964d1471d4d2016d182a20b15ecafcd90c100af97b2a41: Status 404 returned error can't find the container with id 5b1eecb817b23df979964d1471d4d2016d182a20b15ecafcd90c100af97b2a41
Oct 30 21:20:10.382986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.382962    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cptdm\" (UniqueName: \"kubernetes.io/projected/1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04-kube-api-access-cptdm\") pod \"kube-auth-proxy-6645f5c86b-9sbgz\" (UID: \"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04\") " pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.391771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.391754    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"
Oct 30 21:20:10.792585 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:10.792562    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz"]
Oct 30 21:20:10.795164 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:20:10.795134    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e74b7b9_a7cd_4eaf_ba8d_da7805d0ed04.slice/crio-3ca8bd87a4f2f8b11b5b121de51832b59d6c906088f0504f221052b21b33b930 WatchSource:0}: Error finding container 3ca8bd87a4f2f8b11b5b121de51832b59d6c906088f0504f221052b21b33b930: Status 404 returned error can't find the container with id 3ca8bd87a4f2f8b11b5b121de51832b59d6c906088f0504f221052b21b33b930
Oct 30 21:20:11.311726 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:11.311688    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd" event={"ID":"456ba45c-c157-4be2-81fa-a8dcbffcb604","Type":"ContainerStarted","Data":"b64b1bd1e5c7d53446eae9a2f15d7daad6ee5f51ec8d01a3a0bcee3216700f09"}
Oct 30 21:20:11.313033 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:11.312998    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz" event={"ID":"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04","Type":"ContainerStarted","Data":"3ca8bd87a4f2f8b11b5b121de51832b59d6c906088f0504f221052b21b33b930"}
Oct 30 21:20:11.314101 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:11.314075    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j" event={"ID":"86024e40-8bdb-4fcd-b5e8-1cb2045a5925","Type":"ContainerStarted","Data":"5b1eecb817b23df979964d1471d4d2016d182a20b15ecafcd90c100af97b2a41"}
Oct 30 21:20:11.428799 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:11.428742    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-p97rd" podStartSLOduration=2.18674422 podStartE2EDuration="6.428727321s" totalImagesPullingTime="4.241983101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:11.428724786 +0000 UTC m=+346.716447332" watchObservedRunningTime="2026-10-30 21:20:11.428727321 +0000 UTC m=+346.716449861"
Oct 30 21:20:12.322656 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:12.321710    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j" event={"ID":"86024e40-8bdb-4fcd-b5e8-1cb2045a5925","Type":"ContainerStarted","Data":"7ebf227ed79fc4e654ab06e9076fdd2793d2649755df1b50f4c6cb44ca6603c9"}
Oct 30 21:20:12.322656 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:12.321777    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:12.452593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:12.452539    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j" podStartSLOduration=1.599195105 podStartE2EDuration="3.452525235s" totalImagesPullingTime="1.85333013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:12.451326005 +0000 UTC m=+347.739048553" watchObservedRunningTime="2026-10-30 21:20:12.452525235 +0000 UTC m=+347.740247782"
Oct 30 21:20:12.995068 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:12.995024    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-8jqlp"]
Oct 30 21:20:13.004398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.004371    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.053072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.053047    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8jqlp"]
Oct 30 21:20:13.099446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.099412    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sh6pz\""
Oct 30 21:20:13.150207 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.150177    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55c30060-5a44-4bf0-8864-ff90d1db517e-bound-sa-token\") pod \"cert-manager-597b95cb84-8jqlp\" (UID: \"55c30060-5a44-4bf0-8864-ff90d1db517e\") " pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.150392 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.150228    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr6mj\" (UniqueName: \"kubernetes.io/projected/55c30060-5a44-4bf0-8864-ff90d1db517e-kube-api-access-kr6mj\") pod \"cert-manager-597b95cb84-8jqlp\" (UID: \"55c30060-5a44-4bf0-8864-ff90d1db517e\") " pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.251236 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.251151    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55c30060-5a44-4bf0-8864-ff90d1db517e-bound-sa-token\") pod \"cert-manager-597b95cb84-8jqlp\" (UID: \"55c30060-5a44-4bf0-8864-ff90d1db517e\") " pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.251236 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.251228    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr6mj\" (UniqueName: \"kubernetes.io/projected/55c30060-5a44-4bf0-8864-ff90d1db517e-kube-api-access-kr6mj\") pod \"cert-manager-597b95cb84-8jqlp\" (UID: \"55c30060-5a44-4bf0-8864-ff90d1db517e\") " pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.309769 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.309738    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55c30060-5a44-4bf0-8864-ff90d1db517e-bound-sa-token\") pod \"cert-manager-597b95cb84-8jqlp\" (UID: \"55c30060-5a44-4bf0-8864-ff90d1db517e\") " pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.309926 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.309895    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr6mj\" (UniqueName: \"kubernetes.io/projected/55c30060-5a44-4bf0-8864-ff90d1db517e-kube-api-access-kr6mj\") pod \"cert-manager-597b95cb84-8jqlp\" (UID: \"55c30060-5a44-4bf0-8864-ff90d1db517e\") " pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.316667 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.316636    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8jqlp"
Oct 30 21:20:13.696740 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:13.696614    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8jqlp"]
Oct 30 21:20:13.699092 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:20:13.699062    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55c30060_5a44_4bf0_8864_ff90d1db517e.slice/crio-cb6f840ece782ce14735e5cb8474a92f60af402be7f4bcb2f18b2b708953b4ad WatchSource:0}: Error finding container cb6f840ece782ce14735e5cb8474a92f60af402be7f4bcb2f18b2b708953b4ad: Status 404 returned error can't find the container with id cb6f840ece782ce14735e5cb8474a92f60af402be7f4bcb2f18b2b708953b4ad
Oct 30 21:20:14.331057 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:14.331022    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8jqlp" event={"ID":"55c30060-5a44-4bf0-8864-ff90d1db517e","Type":"ContainerStarted","Data":"0d37b33b7e9e103c0b6f3118ba60afdcf92839b009c1ff8ca46cafd5249d01d7"}
Oct 30 21:20:14.331242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:14.331069    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8jqlp" event={"ID":"55c30060-5a44-4bf0-8864-ff90d1db517e","Type":"ContainerStarted","Data":"cb6f840ece782ce14735e5cb8474a92f60af402be7f4bcb2f18b2b708953b4ad"}
Oct 30 21:20:14.332439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:14.332415    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz" event={"ID":"1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04","Type":"ContainerStarted","Data":"e2b9005032c9ec9b8e9780ba79b6c9efc44070682d225493e078c02a087379ca"}
Oct 30 21:20:14.451222 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:14.451178    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-8jqlp" podStartSLOduration=2.4511645619999998 podStartE2EDuration="2.451164562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:14.450742588 +0000 UTC m=+349.738465135" watchObservedRunningTime="2026-10-30 21:20:14.451164562 +0000 UTC m=+349.738887108"
Oct 30 21:20:14.570429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:14.570384    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6645f5c86b-9sbgz" podStartSLOduration=1.498766746 podStartE2EDuration="4.570365366s" totalImagesPullingTime="3.07159862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:14.569543298 +0000 UTC m=+349.857265845" watchObservedRunningTime="2026-10-30 21:20:14.570365366 +0000 UTC m=+349.858087906"
Oct 30 21:20:16.337250 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:20:16.337212    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:20:18.328730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:18.328696    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j"
Oct 30 21:20:45.055503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.055427    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"]
Oct 30 21:20:45.060725 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.060703    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.113142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.113118    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 30 21:20:45.129106 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.129078    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npbcm\" (UniqueName: \"kubernetes.io/projected/cf7aeb0c-a37d-4baf-8a37-034629ebb946-kube-api-access-npbcm\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.129233 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.129126    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf7aeb0c-a37d-4baf-8a37-034629ebb946-metrics-certs\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.129307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.129243    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf7aeb0c-a37d-4baf-8a37-034629ebb946-cert\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.129381 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.129329    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf7aeb0c-a37d-4baf-8a37-034629ebb946-manager-config\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.160609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.160587    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 30 21:20:45.160609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.160605    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 30 21:20:45.160609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.160628    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-5kqlf\""
Oct 30 21:20:45.161815 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.161791    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"]
Oct 30 21:20:45.230281 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.230261    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf7aeb0c-a37d-4baf-8a37-034629ebb946-cert\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.230442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.230304    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf7aeb0c-a37d-4baf-8a37-034629ebb946-manager-config\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.230442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.230421    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npbcm\" (UniqueName: \"kubernetes.io/projected/cf7aeb0c-a37d-4baf-8a37-034629ebb946-kube-api-access-npbcm\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.230559 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.230474    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf7aeb0c-a37d-4baf-8a37-034629ebb946-metrics-certs\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.230991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.230965    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cf7aeb0c-a37d-4baf-8a37-034629ebb946-manager-config\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.232734 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.232711    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf7aeb0c-a37d-4baf-8a37-034629ebb946-metrics-certs\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.232810 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.232756    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf7aeb0c-a37d-4baf-8a37-034629ebb946-cert\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.288892 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.288861    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npbcm\" (UniqueName: \"kubernetes.io/projected/cf7aeb0c-a37d-4baf-8a37-034629ebb946-kube-api-access-npbcm\") pod \"jobset-controller-manager-64f567dd99-6pdqj\" (UID: \"cf7aeb0c-a37d-4baf-8a37-034629ebb946\") " pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.371699 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.371624    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:45.735793 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:45.735768    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"]
Oct 30 21:20:45.739286 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:20:45.739233    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7aeb0c_a37d_4baf_8a37_034629ebb946.slice/crio-7d5a4bec2bb0d3abb09958a6e82cc7ac33c532f1ae29fd93f443ef8aba88a890 WatchSource:0}: Error finding container 7d5a4bec2bb0d3abb09958a6e82cc7ac33c532f1ae29fd93f443ef8aba88a890: Status 404 returned error can't find the container with id 7d5a4bec2bb0d3abb09958a6e82cc7ac33c532f1ae29fd93f443ef8aba88a890
Oct 30 21:20:46.337217 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:20:46.337172    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:20:46.471067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:46.471032    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj" event={"ID":"cf7aeb0c-a37d-4baf-8a37-034629ebb946","Type":"ContainerStarted","Data":"7d5a4bec2bb0d3abb09958a6e82cc7ac33c532f1ae29fd93f443ef8aba88a890"}
Oct 30 21:20:50.490956 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:50.490923    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj" event={"ID":"cf7aeb0c-a37d-4baf-8a37-034629ebb946","Type":"ContainerStarted","Data":"a09dae8ab86c8eb5d6933300046593e06922cd3af7094ac0b290b3b01b9b556c"}
Oct 30 21:20:50.491329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:50.491063    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:20:50.603248 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:20:50.603191    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj" podStartSLOduration=1.51848143 podStartE2EDuration="5.603177002s" totalImagesPullingTime="4.084695572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:50.602192418 +0000 UTC m=+385.889914965" watchObservedRunningTime="2026-10-30 21:20:50.603177002 +0000 UTC m=+385.890899548"
Oct 30 21:21:01.500328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:21:01.500300    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-64f567dd99-6pdqj"
Oct 30 21:21:16.339478 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:21:16.339436    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:21:46.335895 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:21:46.335856    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:22:16.336710 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:22:16.336634    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:22:46.336094 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:22:46.336053    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:23:16.335861 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:23:16.335822    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:23:46.337033 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:23:46.336995    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:24:16.336117 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:24:16.336078    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:24:46.336882 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:24:46.336846    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:25:16.336948 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:25:16.336861    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:25:46.336481 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:25:46.336443    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:26:16.336433 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:26:16.336390    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:26:46.337005 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:26:46.336925    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:27:16.336820 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:27:16.336785    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:27:46.336876 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:27:46.336840    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:28:16.336254 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:28:16.336164    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:28:46.336900 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:28:46.336861    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:29:16.336567 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:29:16.336526    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:29:46.337024 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:29:46.336957    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:30:16.335942 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:30:16.335901    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:30:46.337470 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:30:46.337423    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:30:54.980577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:54.980538    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vlsh8"]
Oct 30 21:30:54.983947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:54.983924    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8"
Oct 30 21:30:55.005749 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.005726    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mftd\" (UniqueName: \"kubernetes.io/projected/6c422031-e721-44cd-a2d8-da9b091c0cd1-kube-api-access-4mftd\") pod \"authorino-56c6c9ddd5-vlsh8\" (UID: \"6c422031-e721-44cd-a2d8-da9b091c0cd1\") " pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8"
Oct 30 21:30:55.036248 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.036227    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:30:55.037391 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.037363    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vlsh8"]
Oct 30 21:30:55.039001 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.038978    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:30:55.083407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.083390    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-klfvt\""
Oct 30 21:30:55.106973 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.106947    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mftd\" (UniqueName: \"kubernetes.io/projected/6c422031-e721-44cd-a2d8-da9b091c0cd1-kube-api-access-4mftd\") pod \"authorino-56c6c9ddd5-vlsh8\" (UID: \"6c422031-e721-44cd-a2d8-da9b091c0cd1\") " pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8"
Oct 30 21:30:55.165791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.165763    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mftd\" (UniqueName: \"kubernetes.io/projected/6c422031-e721-44cd-a2d8-da9b091c0cd1-kube-api-access-4mftd\") pod \"authorino-56c6c9ddd5-vlsh8\" (UID: \"6c422031-e721-44cd-a2d8-da9b091c0cd1\") " pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8"
Oct 30 21:30:55.298558 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.298464    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8"
Oct 30 21:30:55.661494 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.661468    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vlsh8"]
Oct 30 21:30:55.663837 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:30:55.663787    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c422031_e721_44cd_a2d8_da9b091c0cd1.slice/crio-56ae0f8f15a093acb21a20e03bd7903728caf3da8bbb008324b8de555414c60c WatchSource:0}: Error finding container 56ae0f8f15a093acb21a20e03bd7903728caf3da8bbb008324b8de555414c60c: Status 404 returned error can't find the container with id 56ae0f8f15a093acb21a20e03bd7903728caf3da8bbb008324b8de555414c60c
Oct 30 21:30:55.665155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:55.665140    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:30:56.053437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:30:56.053403    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8" event={"ID":"6c422031-e721-44cd-a2d8-da9b091c0cd1","Type":"ContainerStarted","Data":"56ae0f8f15a093acb21a20e03bd7903728caf3da8bbb008324b8de555414c60c"}
Oct 30 21:31:00.072407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:31:00.072376    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8" event={"ID":"6c422031-e721-44cd-a2d8-da9b091c0cd1","Type":"ContainerStarted","Data":"22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee"}
Oct 30 21:31:00.181398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:31:00.181332    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8" podStartSLOduration=2.292414425 podStartE2EDuration="6.181318082s" totalImagesPullingTime="3.888903657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:00.180130225 +0000 UTC m=+995.467852772" watchObservedRunningTime="2026-10-30 21:31:00.181318082 +0000 UTC m=+995.469040633"
Oct 30 21:31:16.337342 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:31:16.337258    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:31:46.337568 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:31:46.337530    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:32:16.336569 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:32:16.336529    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:32:31.197073 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.197029    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-k9w4r"]
Oct 30 21:32:31.199631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.199613    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r"
Oct 30 21:32:31.225429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.225402    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v64zz\" (UniqueName: \"kubernetes.io/projected/2b3419a5-c610-4ede-848e-2dab3d60dbe9-kube-api-access-v64zz\") pod \"authorino-6f6944bcdd-k9w4r\" (UID: \"2b3419a5-c610-4ede-848e-2dab3d60dbe9\") " pod="kuadrant-system/authorino-6f6944bcdd-k9w4r"
Oct 30 21:32:31.254841 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.254818    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-k9w4r"]
Oct 30 21:32:31.326719 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.326685    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v64zz\" (UniqueName: \"kubernetes.io/projected/2b3419a5-c610-4ede-848e-2dab3d60dbe9-kube-api-access-v64zz\") pod \"authorino-6f6944bcdd-k9w4r\" (UID: \"2b3419a5-c610-4ede-848e-2dab3d60dbe9\") " pod="kuadrant-system/authorino-6f6944bcdd-k9w4r"
Oct 30 21:32:31.388248 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.388216    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v64zz\" (UniqueName: \"kubernetes.io/projected/2b3419a5-c610-4ede-848e-2dab3d60dbe9-kube-api-access-v64zz\") pod \"authorino-6f6944bcdd-k9w4r\" (UID: \"2b3419a5-c610-4ede-848e-2dab3d60dbe9\") " pod="kuadrant-system/authorino-6f6944bcdd-k9w4r"
Oct 30 21:32:31.414367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.414322    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"]
Oct 30 21:32:31.416678 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.416661    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.466902 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.466872    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hkklw\""
Oct 30 21:32:31.467134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.467112    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 21:32:31.467204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.467006    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:32:31.475144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.475092    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"]
Oct 30 21:32:31.509835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.509806    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r"
Oct 30 21:32:31.513239 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.513204    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:32:31.513239 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.513223    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:32:31.528389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.528364    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-maas-api-tls\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.528487 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.528402    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tkt7\" (UniqueName: \"kubernetes.io/projected/72d7fcdd-2942-4884-ac28-12fd93421de7-kube-api-access-6tkt7\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.528580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.528555    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-metrics-certs\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.528631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.528601    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d7fcdd-2942-4884-ac28-12fd93421de7-openshift-service-ca\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.598415 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.598384    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-k9w4r"]
Oct 30 21:32:31.629236 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.629210    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-maas-api-tls\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.629399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.629243    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tkt7\" (UniqueName: \"kubernetes.io/projected/72d7fcdd-2942-4884-ac28-12fd93421de7-kube-api-access-6tkt7\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.629399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.629282    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-metrics-certs\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.629399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.629300    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d7fcdd-2942-4884-ac28-12fd93421de7-openshift-service-ca\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.629969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.629947    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d7fcdd-2942-4884-ac28-12fd93421de7-openshift-service-ca\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.631720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.631690    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-metrics-certs\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.631720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.631706    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-maas-api-tls\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.687863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.687835    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tkt7\" (UniqueName: \"kubernetes.io/projected/72d7fcdd-2942-4884-ac28-12fd93421de7-kube-api-access-6tkt7\") pod \"maas-api-86b5466f75-xbqs2\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") " pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.728811 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.728744    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:31.880920 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:31.880893    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-k9w4r"]
Oct 30 21:32:31.883799 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:32:31.883769    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b3419a5_c610_4ede_848e_2dab3d60dbe9.slice/crio-078938b98e40820d6ff0a042835620a02d81d3c1af7281ffb28364a6071c1b2d WatchSource:0}: Error finding container 078938b98e40820d6ff0a042835620a02d81d3c1af7281ffb28364a6071c1b2d: Status 404 returned error can't find the container with id 078938b98e40820d6ff0a042835620a02d81d3c1af7281ffb28364a6071c1b2d
Oct 30 21:32:32.066774 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.066704    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78f598b56c-jj7dz"]
Oct 30 21:32:32.069659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.069638    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.097691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.097667    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"]
Oct 30 21:32:32.100664 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:32:32.100639    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d7fcdd_2942_4884_ac28_12fd93421de7.slice/crio-ed12e0821b1ea7065a4df4b14b21938d9a8eb214ab4f4080e4d7f756f1858566 WatchSource:0}: Error finding container ed12e0821b1ea7065a4df4b14b21938d9a8eb214ab4f4080e4d7f756f1858566: Status 404 returned error can't find the container with id ed12e0821b1ea7065a4df4b14b21938d9a8eb214ab4f4080e4d7f756f1858566
Oct 30 21:32:32.119645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.119626    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 21:32:32.124110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.124088    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f598b56c-jj7dz"]
Oct 30 21:32:32.135074 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.135051    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6205bdb7-c9cd-4858-8884-fd1aef03fc43-tls-cert\") pod \"authorino-78f598b56c-jj7dz\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") " pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.135157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.135101    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8csd\" (UniqueName: \"kubernetes.io/projected/6205bdb7-c9cd-4858-8884-fd1aef03fc43-kube-api-access-d8csd\") pod \"authorino-78f598b56c-jj7dz\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") " pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.236041 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.235976    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6205bdb7-c9cd-4858-8884-fd1aef03fc43-tls-cert\") pod \"authorino-78f598b56c-jj7dz\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") " pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.236041 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.236026    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8csd\" (UniqueName: \"kubernetes.io/projected/6205bdb7-c9cd-4858-8884-fd1aef03fc43-kube-api-access-d8csd\") pod \"authorino-78f598b56c-jj7dz\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") " pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.238331 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.238312    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6205bdb7-c9cd-4858-8884-fd1aef03fc43-tls-cert\") pod \"authorino-78f598b56c-jj7dz\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") " pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.294430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.294399    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8csd\" (UniqueName: \"kubernetes.io/projected/6205bdb7-c9cd-4858-8884-fd1aef03fc43-kube-api-access-d8csd\") pod \"authorino-78f598b56c-jj7dz\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") " pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.379960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.379872    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:32:32.468924 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.468890    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r" event={"ID":"2b3419a5-c610-4ede-848e-2dab3d60dbe9","Type":"ContainerStarted","Data":"078938b98e40820d6ff0a042835620a02d81d3c1af7281ffb28364a6071c1b2d"}
Oct 30 21:32:32.470071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.470045    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2" event={"ID":"72d7fcdd-2942-4884-ac28-12fd93421de7","Type":"ContainerStarted","Data":"ed12e0821b1ea7065a4df4b14b21938d9a8eb214ab4f4080e4d7f756f1858566"}
Oct 30 21:32:32.742915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:32.742706    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78f598b56c-jj7dz"]
Oct 30 21:32:32.745101 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:32:32.745065    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6205bdb7_c9cd_4858_8884_fd1aef03fc43.slice/crio-f38831c0b3ef76fcfe53ec7760a0b010a42af0d23ba9aa137e6dec7cb0085487 WatchSource:0}: Error finding container f38831c0b3ef76fcfe53ec7760a0b010a42af0d23ba9aa137e6dec7cb0085487: Status 404 returned error can't find the container with id f38831c0b3ef76fcfe53ec7760a0b010a42af0d23ba9aa137e6dec7cb0085487
Oct 30 21:32:33.477539 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:33.477499    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f598b56c-jj7dz" event={"ID":"6205bdb7-c9cd-4858-8884-fd1aef03fc43","Type":"ContainerStarted","Data":"f38831c0b3ef76fcfe53ec7760a0b010a42af0d23ba9aa137e6dec7cb0085487"}
Oct 30 21:32:34.488422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.488329    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r" event={"ID":"2b3419a5-c610-4ede-848e-2dab3d60dbe9","Type":"ContainerStarted","Data":"fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193"}
Oct 30 21:32:34.488797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.488528    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r" podUID="2b3419a5-c610-4ede-848e-2dab3d60dbe9" containerName="authorino" containerID="cri-o://fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193" gracePeriod=30
Oct 30 21:32:34.491040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.491011    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f598b56c-jj7dz" event={"ID":"6205bdb7-c9cd-4858-8884-fd1aef03fc43","Type":"ContainerStarted","Data":"72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53"}
Oct 30 21:32:34.600647 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.600594    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r" podStartSLOduration=1.933823092 podStartE2EDuration="3.600575696s" totalImagesPullingTime="1.666752604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:34.598246336 +0000 UTC m=+1089.885968882" watchObservedRunningTime="2026-10-30 21:32:34.600575696 +0000 UTC m=+1089.888298244"
Oct 30 21:32:34.743380 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.743246    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-78f598b56c-jj7dz" podStartSLOduration=1.254533736 podStartE2EDuration="2.743229118s" totalImagesPullingTime="1.488695382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:34.709517269 +0000 UTC m=+1089.997239828" watchObservedRunningTime="2026-10-30 21:32:34.743229118 +0000 UTC m=+1090.030951665"
Oct 30 21:32:34.744791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.744764    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vlsh8"]
Oct 30 21:32:34.745022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.744984    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8" podUID="6c422031-e721-44cd-a2d8-da9b091c0cd1" containerName="authorino" containerID="cri-o://22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee" gracePeriod=30
Oct 30 21:32:34.832923 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.832895    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r"
Oct 30 21:32:34.967325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.967278    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b3419a5-c610-4ede-848e-2dab3d60dbe9-kube-api-access-v64zz\" (UniqueName: \"kubernetes.io/projected/2b3419a5-c610-4ede-848e-2dab3d60dbe9-kube-api-access-v64zz\") pod \"2b3419a5-c610-4ede-848e-2dab3d60dbe9\" (UID: \"2b3419a5-c610-4ede-848e-2dab3d60dbe9\") "
Oct 30 21:32:34.972856 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:34.972808    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3419a5-c610-4ede-848e-2dab3d60dbe9-kube-api-access-v64zz" pod "2b3419a5-c610-4ede-848e-2dab3d60dbe9" (UID: "2b3419a5-c610-4ede-848e-2dab3d60dbe9"). InnerVolumeSpecName "kube-api-access-v64zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:35.008780 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.008749    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8"
Oct 30 21:32:35.069314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.068526    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c422031-e721-44cd-a2d8-da9b091c0cd1-kube-api-access-4mftd\" (UniqueName: \"kubernetes.io/projected/6c422031-e721-44cd-a2d8-da9b091c0cd1-kube-api-access-4mftd\") pod \"6c422031-e721-44cd-a2d8-da9b091c0cd1\" (UID: \"6c422031-e721-44cd-a2d8-da9b091c0cd1\") "
Oct 30 21:32:35.069314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.069115    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v64zz\" (UniqueName: \"kubernetes.io/projected/2b3419a5-c610-4ede-848e-2dab3d60dbe9-kube-api-access-v64zz\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:35.071821 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.071788    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c422031-e721-44cd-a2d8-da9b091c0cd1-kube-api-access-4mftd" pod "6c422031-e721-44cd-a2d8-da9b091c0cd1" (UID: "6c422031-e721-44cd-a2d8-da9b091c0cd1"). InnerVolumeSpecName "kube-api-access-4mftd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:35.170432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.170395    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mftd\" (UniqueName: \"kubernetes.io/projected/6c422031-e721-44cd-a2d8-da9b091c0cd1-kube-api-access-4mftd\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:35.497267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.497229    5817 generic.go:356] "Generic (PLEG): container finished" podID="6c422031-e721-44cd-a2d8-da9b091c0cd1" containerID="22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee" exitCode=0
Oct 30 21:32:35.497705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.497272    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8"
Oct 30 21:32:35.497705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.497317    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8" event={"ID":"6c422031-e721-44cd-a2d8-da9b091c0cd1","Type":"ContainerDied","Data":"22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee"}
Oct 30 21:32:35.497705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.497369    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vlsh8" event={"ID":"6c422031-e721-44cd-a2d8-da9b091c0cd1","Type":"ContainerDied","Data":"56ae0f8f15a093acb21a20e03bd7903728caf3da8bbb008324b8de555414c60c"}
Oct 30 21:32:35.497705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.497388    5817 scope.go:122] "RemoveContainer" containerID="22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee"
Oct 30 21:32:35.498999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.498967    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2" event={"ID":"72d7fcdd-2942-4884-ac28-12fd93421de7","Type":"ContainerStarted","Data":"32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23"}
Oct 30 21:32:35.499142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.499117    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:35.500310 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.500286    5817 generic.go:356] "Generic (PLEG): container finished" podID="2b3419a5-c610-4ede-848e-2dab3d60dbe9" containerID="fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193" exitCode=0
Oct 30 21:32:35.500424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.500319    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r"
Oct 30 21:32:35.500424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.500325    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r" event={"ID":"2b3419a5-c610-4ede-848e-2dab3d60dbe9","Type":"ContainerDied","Data":"fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193"}
Oct 30 21:32:35.500424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.500381    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-k9w4r" event={"ID":"2b3419a5-c610-4ede-848e-2dab3d60dbe9","Type":"ContainerDied","Data":"078938b98e40820d6ff0a042835620a02d81d3c1af7281ffb28364a6071c1b2d"}
Oct 30 21:32:35.506750 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.506638    5817 scope.go:122] "RemoveContainer" containerID="22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee"
Oct 30 21:32:35.506966 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:32:35.506942    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee\": container with ID starting with 22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee not found: ID does not exist" containerID="22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee"
Oct 30 21:32:35.507036 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.506979    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee"} err="failed to get container status \"22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee\": rpc error: code = NotFound desc = could not find container \"22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee\": container with ID starting with 22b7b37ae34b41b7d86fcf0870a238f3d91a289c592a50a96d54d885408edcee not found: ID does not exist"
Oct 30 21:32:35.507036 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.506998    5817 scope.go:122] "RemoveContainer" containerID="fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193"
Oct 30 21:32:35.515794 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.515766    5817 scope.go:122] "RemoveContainer" containerID="fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193"
Oct 30 21:32:35.516037 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:32:35.516019    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193\": container with ID starting with fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193 not found: ID does not exist" containerID="fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193"
Oct 30 21:32:35.516101 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.516048    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193"} err="failed to get container status \"fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193\": rpc error: code = NotFound desc = could not find container \"fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193\": container with ID starting with fb2022e41b4adff1a27ecf8cb6e5f84bc7a95d7957059ab3da1a536211732193 not found: ID does not exist"
Oct 30 21:32:35.657458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.657389    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vlsh8"]
Oct 30 21:32:35.677296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.677269    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vlsh8"]
Oct 30 21:32:35.784976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.784947    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-k9w4r"]
Oct 30 21:32:35.787943 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.787922    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-k9w4r"]
Oct 30 21:32:35.900312 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:35.900270    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2" podStartSLOduration=1.649243841 podStartE2EDuration="4.900258369s" totalImagesPullingTime="3.251014528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:35.898791597 +0000 UTC m=+1091.186514142" watchObservedRunningTime="2026-10-30 21:32:35.900258369 +0000 UTC m=+1091.187980915"
Oct 30 21:32:36.841861 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:36.841819    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b3419a5-c610-4ede-848e-2dab3d60dbe9" path="/var/lib/kubelet/pods/2b3419a5-c610-4ede-848e-2dab3d60dbe9/volumes"
Oct 30 21:32:36.842373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:36.842265    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c422031-e721-44cd-a2d8-da9b091c0cd1" path="/var/lib/kubelet/pods/6c422031-e721-44cd-a2d8-da9b091c0cd1/volumes"
Oct 30 21:32:41.512108 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:32:41.512079    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 21:32:46.335799 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:32:46.335760    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:33:16.336798 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:33:16.336762    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:33:40.561052 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.561021    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"]
Oct 30 21:33:40.561475 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.561393    5817 cpu_manager.go:414] "containerMap: removing container" podUID="2b3419a5-c610-4ede-848e-2dab3d60dbe9" containerName="authorino"
Oct 30 21:33:40.561475 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.561406    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b3419a5-c610-4ede-848e-2dab3d60dbe9" containerName="authorino"
Oct 30 21:33:40.561475 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.561419    5817 cpu_manager.go:414] "containerMap: removing container" podUID="6c422031-e721-44cd-a2d8-da9b091c0cd1" containerName="authorino"
Oct 30 21:33:40.561475 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.561425    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c422031-e721-44cd-a2d8-da9b091c0cd1" containerName="authorino"
Oct 30 21:33:40.561605 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.561501    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b3419a5-c610-4ede-848e-2dab3d60dbe9" containerName="authorino"
Oct 30 21:33:40.561605 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.561509    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c422031-e721-44cd-a2d8-da9b091c0cd1" containerName="authorino"
Oct 30 21:33:40.563468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.563444    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.613880 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.613840    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:33:40.641471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.641443    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.641631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.641475    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5281654a-072a-4d7d-bce5-198d3047aa40-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.641631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.641499    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kxrl\" (UniqueName: \"kubernetes.io/projected/5281654a-072a-4d7d-bce5-198d3047aa40-kube-api-access-8kxrl\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.641724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.641647    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.641724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.641680    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.641812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.641745    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.660205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.660178    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:33:40.660339 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.660282    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6dnp2\""
Oct 30 21:33:40.660430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.660406    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 21:33:40.661721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.661702    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"]
Oct 30 21:33:40.742329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742296    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.742329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742329    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5281654a-072a-4d7d-bce5-198d3047aa40-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.742592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742375    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kxrl\" (UniqueName: \"kubernetes.io/projected/5281654a-072a-4d7d-bce5-198d3047aa40-kube-api-access-8kxrl\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.742592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742423    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.742592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742451    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.742592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742497    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.742881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742850    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.743006 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742914    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.743006 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.742936    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.744601 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.744577    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5281654a-072a-4d7d-bce5-198d3047aa40-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.744847 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.744830    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5281654a-072a-4d7d-bce5-198d3047aa40-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.800643 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.800615    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kxrl\" (UniqueName: \"kubernetes.io/projected/5281654a-072a-4d7d-bce5-198d3047aa40-kube-api-access-8kxrl\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-t8jcx\" (UID: \"5281654a-072a-4d7d-bce5-198d3047aa40\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:40.875213 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:40.875136    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:41.239529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:41.239498    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"]
Oct 30 21:33:41.242861 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:33:41.242832    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5281654a_072a_4d7d_bce5_198d3047aa40.slice/crio-da31e30f88fbefef4b53db0d50b4db4213146a51d85d0df52eb39f1b9b5aca93 WatchSource:0}: Error finding container da31e30f88fbefef4b53db0d50b4db4213146a51d85d0df52eb39f1b9b5aca93: Status 404 returned error can't find the container with id da31e30f88fbefef4b53db0d50b4db4213146a51d85d0df52eb39f1b9b5aca93
Oct 30 21:33:41.791285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:41.791248    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx" event={"ID":"5281654a-072a-4d7d-bce5-198d3047aa40","Type":"ContainerStarted","Data":"da31e30f88fbefef4b53db0d50b4db4213146a51d85d0df52eb39f1b9b5aca93"}
Oct 30 21:33:42.797655 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:42.797615    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx" event={"ID":"5281654a-072a-4d7d-bce5-198d3047aa40","Type":"ContainerStarted","Data":"ca26227e38665f7cf6f22c1184d68903f6795f241c283285c8aa6e6891b4353a"}
Oct 30 21:33:42.875402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:42.875373    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:42.910436 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:42.910384    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx" podStartSLOduration=1.443099916 podStartE2EDuration="2.910369039s" totalImagesPullingTime="1.467269123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:42.909447179 +0000 UTC m=+1158.197169728" watchObservedRunningTime="2026-10-30 21:33:42.910369039 +0000 UTC m=+1158.198091636"
Oct 30 21:33:44.467081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.467044    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"]
Oct 30 21:33:44.469571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.469552    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.543149 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.543110    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 21:33:44.567461 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.567423    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"]
Oct 30 21:33:44.580607 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.580573    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.580784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.580613    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.580784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.580692    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8da95b8-bbb8-4d66-b355-4d99d050bc64-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.580875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.580790    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.580875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.580835    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjd4k\" (UniqueName: \"kubernetes.io/projected/e8da95b8-bbb8-4d66-b355-4d99d050bc64-kube-api-access-hjd4k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.580875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.580858    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.681921 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.681883    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.681947    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjd4k\" (UniqueName: \"kubernetes.io/projected/e8da95b8-bbb8-4d66-b355-4d99d050bc64-kube-api-access-hjd4k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.681971    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.682018    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.682033    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.682183    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8da95b8-bbb8-4d66-b355-4d99d050bc64-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.682394    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.682456    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.682739 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.682707    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.684322 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.684294    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8da95b8-bbb8-4d66-b355-4d99d050bc64-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.684712 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.684694    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8da95b8-bbb8-4d66-b355-4d99d050bc64-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.744223 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.744148    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjd4k\" (UniqueName: \"kubernetes.io/projected/e8da95b8-bbb8-4d66-b355-4d99d050bc64-kube-api-access-hjd4k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r\" (UID: \"e8da95b8-bbb8-4d66-b355-4d99d050bc64\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:44.781624 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:44.781384    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:45.149708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:45.149617    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"]
Oct 30 21:33:45.152875 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:33:45.152839    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8da95b8_bbb8_4d66_b355_4d99d050bc64.slice/crio-d53628ccdcff051402a1110a56a112a9dc2b9e8e1adb6de0ddb38c196bc2e306 WatchSource:0}: Error finding container d53628ccdcff051402a1110a56a112a9dc2b9e8e1adb6de0ddb38c196bc2e306: Status 404 returned error can't find the container with id d53628ccdcff051402a1110a56a112a9dc2b9e8e1adb6de0ddb38c196bc2e306
Oct 30 21:33:45.815372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:45.814554    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r" event={"ID":"e8da95b8-bbb8-4d66-b355-4d99d050bc64","Type":"ContainerStarted","Data":"7fbfd306c92367fb518914602ce40e491e389a84962e435c8104977ca8991fa2"}
Oct 30 21:33:45.815372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:45.814621    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r" event={"ID":"e8da95b8-bbb8-4d66-b355-4d99d050bc64","Type":"ContainerStarted","Data":"d53628ccdcff051402a1110a56a112a9dc2b9e8e1adb6de0ddb38c196bc2e306"}
Oct 30 21:33:45.927867 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:45.927815    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r" podStartSLOduration=1.354041928 podStartE2EDuration="1.927800581s" totalImagesPullingTime="573.758653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:45.927056634 +0000 UTC m=+1161.214779179" watchObservedRunningTime="2026-10-30 21:33:45.927800581 +0000 UTC m=+1161.215523128"
Oct 30 21:33:46.336766 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:33:46.336726    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:33:46.781698 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:46.781662    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:50.002800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.002766    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"]
Oct 30 21:33:50.006373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.006335    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.043457 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.043423    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.043587 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.043475    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.043653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.043589    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.043653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.043611    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htm4k\" (UniqueName: \"kubernetes.io/projected/41f13ead-4edc-44a9-a810-8eb47a1b51f2-kube-api-access-htm4k\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.043653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.043628    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.043803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.043662    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f13ead-4edc-44a9-a810-8eb47a1b51f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.107842 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.107809    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 30 21:33:50.109188 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.109162    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"]
Oct 30 21:33:50.144919 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.144887    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145069 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.144932    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htm4k\" (UniqueName: \"kubernetes.io/projected/41f13ead-4edc-44a9-a810-8eb47a1b51f2-kube-api-access-htm4k\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145069 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.144962    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145069 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.145009    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f13ead-4edc-44a9-a810-8eb47a1b51f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145239 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.145056    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145239 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.145114    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.145464    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145573 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.145491    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.145573 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.145528    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.147123 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.147102    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f13ead-4edc-44a9-a810-8eb47a1b51f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.147462 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.147447    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f13ead-4edc-44a9-a810-8eb47a1b51f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.202845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.202814    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htm4k\" (UniqueName: \"kubernetes.io/projected/41f13ead-4edc-44a9-a810-8eb47a1b51f2-kube-api-access-htm4k\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v\" (UID: \"41f13ead-4edc-44a9-a810-8eb47a1b51f2\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.318576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.318482    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:50.687056 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.685537    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"]
Oct 30 21:33:50.692676 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:33:50.692649    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f13ead_4edc_44a9_a810_8eb47a1b51f2.slice/crio-e500e7772069196bc046b91d0656329c4835e2383e288632296e148c608ca9cc WatchSource:0}: Error finding container e500e7772069196bc046b91d0656329c4835e2383e288632296e148c608ca9cc: Status 404 returned error can't find the container with id e500e7772069196bc046b91d0656329c4835e2383e288632296e148c608ca9cc
Oct 30 21:33:50.841117 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.841084    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v" event={"ID":"41f13ead-4edc-44a9-a810-8eb47a1b51f2","Type":"ContainerStarted","Data":"e500e7772069196bc046b91d0656329c4835e2383e288632296e148c608ca9cc"}
Oct 30 21:33:50.875488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.875462    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:50.887697 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:50.887672    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:51.844579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:51.844547    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v" event={"ID":"41f13ead-4edc-44a9-a810-8eb47a1b51f2","Type":"ContainerStarted","Data":"397ec6849d675db30f7c8cb3942f40fe2b9ecaa4325bc47aaf0c3759c7852ab9"}
Oct 30 21:33:51.855747 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:51.855717    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-t8jcx"
Oct 30 21:33:51.957447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:51.957399    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v" podStartSLOduration=2.392816942 podStartE2EDuration="2.957385569s" totalImagesPullingTime="564.568627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:51.956704639 +0000 UTC m=+1167.244427185" watchObservedRunningTime="2026-10-30 21:33:51.957385569 +0000 UTC m=+1167.245108109"
Oct 30 21:33:52.318838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:52.318806    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:33:54.781967 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:54.781930    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:54.794748 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:54.794721    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:54.869329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:54.869300    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r"
Oct 30 21:33:57.657874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.657834    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"]
Oct 30 21:33:57.662048 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.662022    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.711681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.711655    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:33:57.714115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714087    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714233 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714130    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714233 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714153    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714233 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714191    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714381 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714310    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714381 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714371    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714460    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqv42\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-kube-api-access-nqv42\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714483    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.714523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.714498    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.758390 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.758365    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z4vl5\""
Oct 30 21:33:57.759995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.759972    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"]
Oct 30 21:33:57.815852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.815828    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.815981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.815861    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.815981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.815889    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqv42\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-kube-api-access-nqv42\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.815981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.815906    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.815981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.815930    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816149 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816015    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816149 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816053    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816149 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816081    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816149 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816103    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816341    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816486 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816397    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816573    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816647 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816593    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.816799 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.816777    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.818243 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.818223    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.818456 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.818438    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.876108 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.876080    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.876108 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.876099    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqv42\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-kube-api-access-nqv42\") pod \"maas-default-gateway-openshift-default-67bf986f99-lf929\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:57.974014 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:57.973984    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:33:58.388186 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:58.388161    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"]
Oct 30 21:33:58.878558 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:33:58.878522    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" event={"ID":"0c44396d-d8fd-417b-bf4e-95b1b5256e76","Type":"ContainerStarted","Data":"bc023f43f6b724fa23693f58762913d14293a9fecfb8dde0239f532472fe18ac"}
Oct 30 21:34:00.319136 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:00.319100    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:34:00.335837 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:00.335806    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:34:00.904438 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:00.904405    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v"
Oct 30 21:34:02.299680 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:02.299645    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:34:02.299950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:02.299716    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:34:02.299950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:02.299751    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:34:02.901976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:02.901937    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" event={"ID":"0c44396d-d8fd-417b-bf4e-95b1b5256e76","Type":"ContainerStarted","Data":"021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7"}
Oct 30 21:34:02.974995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:02.974960    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:34:03.058046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:03.057991    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podStartSLOduration=2.152109981 podStartE2EDuration="6.057979348s" totalImagesPullingTime="3.905869367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:03.057358615 +0000 UTC m=+1178.345081153" watchObservedRunningTime="2026-10-30 21:34:03.057979348 +0000 UTC m=+1178.345701894"
Oct 30 21:34:03.976058 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:03.976023    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-lf929 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:34:03.976516 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:03.976079    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:34:04.974308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:04.974273    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-lf929 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:34:04.974493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:04.974325    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:34:05.979442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:05.979404    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-lf929 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:34:05.979895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:05.979468    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:34:06.974669 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:06.974627    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-lf929 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:34:06.974815 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:06.974678    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:34:07.974504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:07.974473    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-lf929 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:34:07.974882 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:07.974527    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:34:08.974293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:08.974255    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-lf929 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:34:08.974474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:08.974319    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:34:09.974813 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:09.974781    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-lf929 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Oct 30 21:34:09.975192 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:09.974845    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Oct 30 21:34:10.978902 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:10.978875    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:34:10.979625 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:10.979128    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:34:10.979887 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:10.979868    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:34:16.336596 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:34:16.336561    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:34:21.788968 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.788932    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86978f68d-xztz5"]
Oct 30 21:34:21.792625 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.792603    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:21.847010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.846981    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86978f68d-xztz5"]
Oct 30 21:34:21.876433 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.876397    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/efa553c6-4d31-47f8-8baf-d38186d20bae-tls-cert\") pod \"authorino-86978f68d-xztz5\" (UID: \"efa553c6-4d31-47f8-8baf-d38186d20bae\") " pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:21.876598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.876541    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzr4s\" (UniqueName: \"kubernetes.io/projected/efa553c6-4d31-47f8-8baf-d38186d20bae-kube-api-access-hzr4s\") pod \"authorino-86978f68d-xztz5\" (UID: \"efa553c6-4d31-47f8-8baf-d38186d20bae\") " pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:21.977425 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.977385    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/efa553c6-4d31-47f8-8baf-d38186d20bae-tls-cert\") pod \"authorino-86978f68d-xztz5\" (UID: \"efa553c6-4d31-47f8-8baf-d38186d20bae\") " pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:21.977609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.977483    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzr4s\" (UniqueName: \"kubernetes.io/projected/efa553c6-4d31-47f8-8baf-d38186d20bae-kube-api-access-hzr4s\") pod \"authorino-86978f68d-xztz5\" (UID: \"efa553c6-4d31-47f8-8baf-d38186d20bae\") " pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:21.979845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:21.979823    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/efa553c6-4d31-47f8-8baf-d38186d20bae-tls-cert\") pod \"authorino-86978f68d-xztz5\" (UID: \"efa553c6-4d31-47f8-8baf-d38186d20bae\") " pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:22.036477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:22.036441    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzr4s\" (UniqueName: \"kubernetes.io/projected/efa553c6-4d31-47f8-8baf-d38186d20bae-kube-api-access-hzr4s\") pod \"authorino-86978f68d-xztz5\" (UID: \"efa553c6-4d31-47f8-8baf-d38186d20bae\") " pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:22.104296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:22.104216    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86978f68d-xztz5"
Oct 30 21:34:22.470720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:22.470694    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86978f68d-xztz5"]
Oct 30 21:34:22.473321 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:34:22.473284    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefa553c6_4d31_47f8_8baf_d38186d20bae.slice/crio-bfa624030051a4b9ed42758f0d3978002fce44cb919fe45abe41c8443debf3b4 WatchSource:0}: Error finding container bfa624030051a4b9ed42758f0d3978002fce44cb919fe45abe41c8443debf3b4: Status 404 returned error can't find the container with id bfa624030051a4b9ed42758f0d3978002fce44cb919fe45abe41c8443debf3b4
Oct 30 21:34:23.003045 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:23.003006    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86978f68d-xztz5" event={"ID":"efa553c6-4d31-47f8-8baf-d38186d20bae","Type":"ContainerStarted","Data":"bfa624030051a4b9ed42758f0d3978002fce44cb919fe45abe41c8443debf3b4"}
Oct 30 21:34:25.016649 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.016608    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86978f68d-xztz5" event={"ID":"efa553c6-4d31-47f8-8baf-d38186d20bae","Type":"ContainerStarted","Data":"fdc48246c443e52126a773296e2f44d9bd8742363550c1e90af7587962221beb"}
Oct 30 21:34:25.128286 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.128230    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-86978f68d-xztz5" podStartSLOduration=2.350019709 podStartE2EDuration="4.128215506s" totalImagesPullingTime="1.778195797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:25.126487059 +0000 UTC m=+1200.414209606" watchObservedRunningTime="2026-10-30 21:34:25.128215506 +0000 UTC m=+1200.415938052"
Oct 30 21:34:25.177505 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.175681    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f598b56c-jj7dz"]
Oct 30 21:34:25.177505 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.175901    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-78f598b56c-jj7dz" podUID="6205bdb7-c9cd-4858-8884-fd1aef03fc43" containerName="authorino" containerID="cri-o://72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53" gracePeriod=30
Oct 30 21:34:25.429355 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.429318    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:34:25.526669 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.526633    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6205bdb7-c9cd-4858-8884-fd1aef03fc43-tls-cert\" (UniqueName: \"kubernetes.io/secret/6205bdb7-c9cd-4858-8884-fd1aef03fc43-tls-cert\") pod \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") "
Oct 30 21:34:25.526838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.526799    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6205bdb7-c9cd-4858-8884-fd1aef03fc43-kube-api-access-d8csd\" (UniqueName: \"kubernetes.io/projected/6205bdb7-c9cd-4858-8884-fd1aef03fc43-kube-api-access-d8csd\") pod \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\" (UID: \"6205bdb7-c9cd-4858-8884-fd1aef03fc43\") "
Oct 30 21:34:25.528760 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.528732    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6205bdb7-c9cd-4858-8884-fd1aef03fc43-kube-api-access-d8csd" pod "6205bdb7-c9cd-4858-8884-fd1aef03fc43" (UID: "6205bdb7-c9cd-4858-8884-fd1aef03fc43"). InnerVolumeSpecName "kube-api-access-d8csd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:25.537483 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.537457    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6205bdb7-c9cd-4858-8884-fd1aef03fc43-tls-cert" pod "6205bdb7-c9cd-4858-8884-fd1aef03fc43" (UID: "6205bdb7-c9cd-4858-8884-fd1aef03fc43"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:25.627547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.627511    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8csd\" (UniqueName: \"kubernetes.io/projected/6205bdb7-c9cd-4858-8884-fd1aef03fc43-kube-api-access-d8csd\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:25.627547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:25.627541    5817 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6205bdb7-c9cd-4858-8884-fd1aef03fc43-tls-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:26.024193 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.024159    5817 generic.go:356] "Generic (PLEG): container finished" podID="6205bdb7-c9cd-4858-8884-fd1aef03fc43" containerID="72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53" exitCode=0
Oct 30 21:34:26.024601 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.024212    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78f598b56c-jj7dz"
Oct 30 21:34:26.024601 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.024237    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f598b56c-jj7dz" event={"ID":"6205bdb7-c9cd-4858-8884-fd1aef03fc43","Type":"ContainerDied","Data":"72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53"}
Oct 30 21:34:26.024601 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.024284    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78f598b56c-jj7dz" event={"ID":"6205bdb7-c9cd-4858-8884-fd1aef03fc43","Type":"ContainerDied","Data":"f38831c0b3ef76fcfe53ec7760a0b010a42af0d23ba9aa137e6dec7cb0085487"}
Oct 30 21:34:26.024601 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.024304    5817 scope.go:122] "RemoveContainer" containerID="72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53"
Oct 30 21:34:26.034830 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.034805    5817 scope.go:122] "RemoveContainer" containerID="72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53"
Oct 30 21:34:26.035095 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:34:26.035074    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53\": container with ID starting with 72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53 not found: ID does not exist" containerID="72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53"
Oct 30 21:34:26.035177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.035099    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53"} err="failed to get container status \"72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53\": rpc error: code = NotFound desc = could not find container \"72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53\": container with ID starting with 72bed72133bcd887e7e5b8314b9b090b2d2eeecd611b517bd19565317705ee53 not found: ID does not exist"
Oct 30 21:34:26.178991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.178953    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78f598b56c-jj7dz"]
Oct 30 21:34:26.182895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.182871    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78f598b56c-jj7dz"]
Oct 30 21:34:26.841565 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:26.841522    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6205bdb7-c9cd-4858-8884-fd1aef03fc43" path="/var/lib/kubelet/pods/6205bdb7-c9cd-4858-8884-fd1aef03fc43/volumes"
Oct 30 21:34:27.671278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.671250    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"]
Oct 30 21:34:27.671685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.671663    5817 cpu_manager.go:414] "containerMap: removing container" podUID="6205bdb7-c9cd-4858-8884-fd1aef03fc43" containerName="authorino"
Oct 30 21:34:27.671685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.671675    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6205bdb7-c9cd-4858-8884-fd1aef03fc43" containerName="authorino"
Oct 30 21:34:27.671784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.671773    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="6205bdb7-c9cd-4858-8884-fd1aef03fc43" containerName="authorino"
Oct 30 21:34:27.678214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.677158    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759213 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759175    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759287    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759402    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759525 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759457    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759565 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759542    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759641    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759694    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759760 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759737    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsv5q\" (UniqueName: \"kubernetes.io/projected/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-kube-api-access-dsv5q\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.759875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.759846    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.775885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.775856    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"]
Oct 30 21:34:27.860727 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860689    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.860903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860745    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.860903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860782    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsv5q\" (UniqueName: \"kubernetes.io/projected/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-kube-api-access-dsv5q\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.860903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860822    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.860903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860861    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.860903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860899    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860943    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.860995    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.861026    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861376 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.861281    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861448 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.861387    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.861483    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.861636    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.861761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.861741    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.863633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.863603    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.863633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.863628    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.918718 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.918686    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.918859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.918699    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsv5q\" (UniqueName: \"kubernetes.io/projected/a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878-kube-api-access-dsv5q\") pod \"maas-default-gateway-openshift-default-67bf986f99-hkq67\" (UID: \"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:27.990952 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:27.990917    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:28.408746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:28.408670    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"]
Oct 30 21:34:28.411909 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:34:28.411874    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32c9c9e_9f2c_4b26_ad2f_7f81b04fd878.slice/crio-6de63c6ce9bf04873a848cf33b3701f34d852e331c30bb0c4d7e540295187dab WatchSource:0}: Error finding container 6de63c6ce9bf04873a848cf33b3701f34d852e331c30bb0c4d7e540295187dab: Status 404 returned error can't find the container with id 6de63c6ce9bf04873a848cf33b3701f34d852e331c30bb0c4d7e540295187dab
Oct 30 21:34:28.414044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:28.414008    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:34:28.414143 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:28.414095    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:34:28.414209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:28.414142    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:34:29.041859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:29.041826    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" event={"ID":"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878","Type":"ContainerStarted","Data":"65c4a73f1ee009914e1e87941ad51c155597f087c5c78e41099dea54e9b64982"}
Oct 30 21:34:29.042258 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:29.041888    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" event={"ID":"a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878","Type":"ContainerStarted","Data":"6de63c6ce9bf04873a848cf33b3701f34d852e331c30bb0c4d7e540295187dab"}
Oct 30 21:34:29.228387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:29.228311    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podStartSLOduration=2.228297533 podStartE2EDuration="2.228297533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:29.227993051 +0000 UTC m=+1204.515715597" watchObservedRunningTime="2026-10-30 21:34:29.228297533 +0000 UTC m=+1204.516020078"
Oct 30 21:34:30.001934 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:29.994616    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:30.011376 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:30.010485    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:34:30.011376 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:30.010544    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:34:30.993371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:30.992725    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:34:30.993371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:30.992789    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:34:31.999371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:31.992630    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:34:31.999371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:31.992690    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:34:33.002372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:32.997686    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:34:33.002372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:32.997748    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:34:34.000373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:33.993645    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:34:34.000373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:33.993713    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:34:34.992046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:34.992009    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:34:34.992446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:34.992067    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:34:36.991957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:36.991917    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:34:36.992496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:36.991982    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:34:37.435428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.435322    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-69c889db49-4gj8j"]
Oct 30 21:34:37.441161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.441135    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:37.492459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.492422    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:34:37.492459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.492431    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:34:37.492669 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.492481    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 21:34:37.539251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.539213    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 21:34:37.539512 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.539369    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-sszvx\""
Oct 30 21:34:37.540764 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.540734    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-69c889db49-4gj8j"]
Oct 30 21:34:37.581795 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.581767    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13f4baa5-f7c2-45ff-955e-60c8722193d0-metrics-certs\") pod \"ai-gateway-controller-69c889db49-4gj8j\" (UID: \"13f4baa5-f7c2-45ff-955e-60c8722193d0\") " pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:37.581884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.581826    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkf7q\" (UniqueName: \"kubernetes.io/projected/13f4baa5-f7c2-45ff-955e-60c8722193d0-kube-api-access-vkf7q\") pod \"ai-gateway-controller-69c889db49-4gj8j\" (UID: \"13f4baa5-f7c2-45ff-955e-60c8722193d0\") " pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:37.682970 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.682938    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13f4baa5-f7c2-45ff-955e-60c8722193d0-metrics-certs\") pod \"ai-gateway-controller-69c889db49-4gj8j\" (UID: \"13f4baa5-f7c2-45ff-955e-60c8722193d0\") " pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:37.683119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.683000    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkf7q\" (UniqueName: \"kubernetes.io/projected/13f4baa5-f7c2-45ff-955e-60c8722193d0-kube-api-access-vkf7q\") pod \"ai-gateway-controller-69c889db49-4gj8j\" (UID: \"13f4baa5-f7c2-45ff-955e-60c8722193d0\") " pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:37.685506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.685453    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13f4baa5-f7c2-45ff-955e-60c8722193d0-metrics-certs\") pod \"ai-gateway-controller-69c889db49-4gj8j\" (UID: \"13f4baa5-f7c2-45ff-955e-60c8722193d0\") " pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:37.741754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.741714    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkf7q\" (UniqueName: \"kubernetes.io/projected/13f4baa5-f7c2-45ff-955e-60c8722193d0-kube-api-access-vkf7q\") pod \"ai-gateway-controller-69c889db49-4gj8j\" (UID: \"13f4baa5-f7c2-45ff-955e-60c8722193d0\") " pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:37.755612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:37.755573    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:38.033544 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:38.033501    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-hkq67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:34:38.034017 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:38.033574    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67" podUID="a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:34:38.126848 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:38.126818    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-69c889db49-4gj8j"]
Oct 30 21:34:38.131231 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:34:38.131200    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f4baa5_f7c2_45ff_955e_60c8722193d0.slice/crio-0db89f934e9105525d8ee0b7a75f8e1cd5f0ebfd462fddbec5a35ce4ca561c2c WatchSource:0}: Error finding container 0db89f934e9105525d8ee0b7a75f8e1cd5f0ebfd462fddbec5a35ce4ca561c2c: Status 404 returned error can't find the container with id 0db89f934e9105525d8ee0b7a75f8e1cd5f0ebfd462fddbec5a35ce4ca561c2c
Oct 30 21:34:38.642443 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:38.642407    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:38.642767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:38.642731    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:38.648905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:38.648884    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-hkq67"
Oct 30 21:34:39.090746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:39.090705    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j" event={"ID":"13f4baa5-f7c2-45ff-955e-60c8722193d0","Type":"ContainerStarted","Data":"0db89f934e9105525d8ee0b7a75f8e1cd5f0ebfd462fddbec5a35ce4ca561c2c"}
Oct 30 21:34:41.103634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:41.103600    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j" event={"ID":"13f4baa5-f7c2-45ff-955e-60c8722193d0","Type":"ContainerStarted","Data":"eda122bd585d8d9967dace7aaea5c6efe42bd90b0359368c537b61c79d621bc3"}
Oct 30 21:34:41.104009 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:41.103724    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:34:41.214908 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:41.214864    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j" podStartSLOduration=2.134790879 podStartE2EDuration="4.214850652s" totalImagesPullingTime="2.080059773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:41.214240615 +0000 UTC m=+1216.501963162" watchObservedRunningTime="2026-10-30 21:34:41.214850652 +0000 UTC m=+1216.502573198"
Oct 30 21:34:46.337207 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:34:46.337166    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:34:52.110964 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:34:52.110929    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-69c889db49-4gj8j"
Oct 30 21:35:16.336332 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:35:16.336296    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:35:27.315166 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.315131    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"]
Oct 30 21:35:27.320359 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.320327    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.370240 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.370214    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-554f46-openshift-default-dockercfg-lpld4\""
Oct 30 21:35:27.418891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.418867    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"]
Oct 30 21:35:27.463423 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463390    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463423 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463423    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463591 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463446    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhg2x\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-kube-api-access-bhg2x\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463591 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463576    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463662 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463628    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463695 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463665    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463729 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463693    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463753    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.463806 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.463787    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.564885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.564856    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.564916    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.564946    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.564977    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.564999    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565270 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565036    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565270 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565119    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565270 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565144    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565270 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565180    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhg2x\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-kube-api-access-bhg2x\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565421    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565468    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565628 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565605    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565632    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.565726 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.565698    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.567429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.567403    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.567628 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.567611    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.624961 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.624934    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.625081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.625063    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhg2x\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-kube-api-access-bhg2x\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.632860 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.632833    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:27.906228 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.906162    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"]
Oct 30 21:35:27.910974 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.910952    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:27.961319 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.961295    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-a2dced-openshift-default-dockercfg-ll74c\""
Oct 30 21:35:27.971253 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:27.971222    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"]
Oct 30 21:35:28.065095 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.065071    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"]
Oct 30 21:35:28.068593 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:28.068567    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f80f7d_26be_43b8_b355_9c7e8a9924d7.slice/crio-50360112bd9d954a7744eae3546d9bd0fffb68a71b6d013e4860f153ffe4514e WatchSource:0}: Error finding container 50360112bd9d954a7744eae3546d9bd0fffb68a71b6d013e4860f153ffe4514e: Status 404 returned error can't find the container with id 50360112bd9d954a7744eae3546d9bd0fffb68a71b6d013e4860f153ffe4514e
Oct 30 21:35:28.070832 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070803    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.070892 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070818    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:28.070892 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070855    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.070960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070888    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:28.070960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070918    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:28.070960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070940    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.071070 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070964    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.071070 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.070987    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq6l2\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-kube-api-access-zq6l2\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.071204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.071112    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.071204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.071168    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.071279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.071231    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.071279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.071266    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172532    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172586    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172625    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172653    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172679    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6l2\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-kube-api-access-zq6l2\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172711    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172759    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.172812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172795    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.173130 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172834    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.173130 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.172998    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.173130 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.173026    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.173130 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.173115    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.173341 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.173257    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.173679 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.173659    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.175227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.175203    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.175625 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.175602    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.233063 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.233036    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.233215 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.233072    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6l2\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-kube-api-access-zq6l2\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.342826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.342100    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" event={"ID":"f7f80f7d-26be-43b8-b355-9c7e8a9924d7","Type":"ContainerStarted","Data":"b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb"}
Oct 30 21:35:28.342826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.342160    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" event={"ID":"f7f80f7d-26be-43b8-b355-9c7e8a9924d7","Type":"ContainerStarted","Data":"50360112bd9d954a7744eae3546d9bd0fffb68a71b6d013e4860f153ffe4514e"}
Oct 30 21:35:28.490238 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.490181    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podStartSLOduration=1.490161462 podStartE2EDuration="1.490161462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:28.486067132 +0000 UTC m=+1263.773789678" watchObservedRunningTime="2026-10-30 21:35:28.490161462 +0000 UTC m=+1263.777884010"
Oct 30 21:35:28.524594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.524563    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:28.598652 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.598621    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"]
Oct 30 21:35:28.603517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.603492    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.634007 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.633979    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:28.653938 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.653912    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-afbe5b-openshift-default-dockercfg-jbdjs\""
Oct 30 21:35:28.661906 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.661880    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"]
Oct 30 21:35:28.780721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.780653    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.780721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.780700    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.780721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.780718    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.780929 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.780739    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwpq8\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-kube-api-access-fwpq8\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.780982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.780910    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.780982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.780952    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.780982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.780974    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.781099 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.781084    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.781139 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.781108    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.882536 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882503    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.882703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882540    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.882703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882568    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwpq8\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-kube-api-access-fwpq8\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.882703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882648    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.882703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882673    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.882703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882697    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.882963 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882899    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.883020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.882962    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.883020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.883011    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.883263 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.883203    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.883263 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.883211    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.883482 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.883450    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.883545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.883525    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.883673 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.883655    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.885147 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.885125    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.885331 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.885313    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.941535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.941507    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwpq8\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-kube-api-access-fwpq8\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.943174 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.943133    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:28.943576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.943555    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"]
Oct 30 21:35:28.945067 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:28.945036    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d54ac74_962b_4549_8c9b_a2c98f4d3251.slice/crio-ac4aee690dc77ef4a7a34b65b46b57dc2458e511c47b58fbbe8a4b198af27957 WatchSource:0}: Error finding container ac4aee690dc77ef4a7a34b65b46b57dc2458e511c47b58fbbe8a4b198af27957: Status 404 returned error can't find the container with id ac4aee690dc77ef4a7a34b65b46b57dc2458e511c47b58fbbe8a4b198af27957
Oct 30 21:35:28.947766 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.947730    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:28.947870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.947811    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:28.947870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:28.947846    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:29.195672 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.195594    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"]
Oct 30 21:35:29.199362 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.199317    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.229551 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.229380    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:29.248982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.248959    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-991335-openshift-default-dockercfg-xsdjt\""
Oct 30 21:35:29.263440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.263414    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"]
Oct 30 21:35:29.352013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.351887    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" event={"ID":"9d54ac74-962b-4549-8c9b-a2c98f4d3251","Type":"ContainerStarted","Data":"41e1149c1a6943960b443842ebbd9b9838247375d37f8e340225d68296b25e03"}
Oct 30 21:35:29.352013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.351967    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" event={"ID":"9d54ac74-962b-4549-8c9b-a2c98f4d3251","Type":"ContainerStarted","Data":"ac4aee690dc77ef4a7a34b65b46b57dc2458e511c47b58fbbe8a4b198af27957"}
Oct 30 21:35:29.386600 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386568    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.386600 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386605    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.386808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386737    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.386808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386775    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff45733c-4a30-43c1-b509-18792c82ec04-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.386808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386800    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.386920 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386895    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2x7l\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-kube-api-access-d2x7l\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.386953 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386921    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.386953 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386941    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.387024 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.386964    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff45733c-4a30-43c1-b509-18792c82ec04-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.469475 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.469431    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podStartSLOduration=2.469418603 podStartE2EDuration="2.469418603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:29.467742089 +0000 UTC m=+1264.755464635" watchObservedRunningTime="2026-10-30 21:35:29.469418603 +0000 UTC m=+1264.757141148"
Oct 30 21:35:29.488434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488406    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488584 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488460    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488584 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488566    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488702 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488602    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff45733c-4a30-43c1-b509-18792c82ec04-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488702 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488652    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488781    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2x7l\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-kube-api-access-d2x7l\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488851    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488927 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488888    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.488927 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488889    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.489030 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488955    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff45733c-4a30-43c1-b509-18792c82ec04-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.489084 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.489054    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.489144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.489113    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.489206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.488951    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.489824 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.489801    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff45733c-4a30-43c1-b509-18792c82ec04-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.490767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.490742    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.491284 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.491262    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff45733c-4a30-43c1-b509-18792c82ec04-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.525415 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.525396    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:29.547020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.546999    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.547148 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.547131    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2x7l\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-kube-api-access-d2x7l\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:29.618198 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.618172    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"]
Oct 30 21:35:29.621794 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:29.621755    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8f0730_9523_4719_a5d9_f3aee98bb8b5.slice/crio-20068f32af37b3d1cf5dbf6f04c969c53bba956decff1eeabc7da79f3342aa05 WatchSource:0}: Error finding container 20068f32af37b3d1cf5dbf6f04c969c53bba956decff1eeabc7da79f3342aa05: Status 404 returned error can't find the container with id 20068f32af37b3d1cf5dbf6f04c969c53bba956decff1eeabc7da79f3342aa05
Oct 30 21:35:29.624330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.624291    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:29.624444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.624399    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:29.624484 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.624459    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:29.638991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.638971    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:29.812748 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:29.812668    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:30.188675 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.188649    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"]
Oct 30 21:35:30.190665 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:30.190610    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff45733c_4a30_43c1_b509_18792c82ec04.slice/crio-bc6af06dd1cbc060bf49728eb89b5e27cd257fbe3e3b2bb30b80c7ff7fbb33e1 WatchSource:0}: Error finding container bc6af06dd1cbc060bf49728eb89b5e27cd257fbe3e3b2bb30b80c7ff7fbb33e1: Status 404 returned error can't find the container with id bc6af06dd1cbc060bf49728eb89b5e27cd257fbe3e3b2bb30b80c7ff7fbb33e1
Oct 30 21:35:30.193108 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.193072    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:30.193194 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.193141    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:30.193194 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.193174    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:30.358422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.358379    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" event={"ID":"2c8f0730-9523-4719-a5d9-f3aee98bb8b5","Type":"ContainerStarted","Data":"7bb86c03431c663a0c69b12e7c4231250d7959738ac5a2b863be2177a8b736e1"}
Oct 30 21:35:30.358828 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.358436    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" event={"ID":"2c8f0730-9523-4719-a5d9-f3aee98bb8b5","Type":"ContainerStarted","Data":"20068f32af37b3d1cf5dbf6f04c969c53bba956decff1eeabc7da79f3342aa05"}
Oct 30 21:35:30.360014 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.359992    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" event={"ID":"ff45733c-4a30-43c1-b509-18792c82ec04","Type":"ContainerStarted","Data":"38019071fc0d63dce06612171d26127afcc6b1c2595983311f25c9d7ea9ace40"}
Oct 30 21:35:30.360096 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.360033    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" event={"ID":"ff45733c-4a30-43c1-b509-18792c82ec04","Type":"ContainerStarted","Data":"bc6af06dd1cbc060bf49728eb89b5e27cd257fbe3e3b2bb30b80c7ff7fbb33e1"}
Oct 30 21:35:30.360300 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.360285    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:30.361169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.361148    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:30.476092 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.476042    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" podStartSLOduration=2.476028434 podStartE2EDuration="2.476028434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:30.474368053 +0000 UTC m=+1265.762090601" watchObservedRunningTime="2026-10-30 21:35:30.476028434 +0000 UTC m=+1265.763750982"
Oct 30 21:35:30.529702 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.529674    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:30.711732 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.711678    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podStartSLOduration=1.7116650519999999 podStartE2EDuration="1.711665052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:30.710335164 +0000 UTC m=+1265.998057710" watchObservedRunningTime="2026-10-30 21:35:30.711665052 +0000 UTC m=+1265.999387597"
Oct 30 21:35:30.813327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:30.813260    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:31.229659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:31.229632    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:31.234543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:31.234522    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:31.365189 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:31.365165    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:31.365590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:31.365210    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:31.366202 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:31.366167    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:35:31.366320 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:31.366237    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:35:31.817510 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:31.817486    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:32.370114 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:32.370083    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:32.370974 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:32.370952    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:35:41.550868 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.550835    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"]
Oct 30 21:35:41.554279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.554258    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.603791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.603768    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-554f46-dockercfg-gn64k\""
Oct 30 21:35:41.619289 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.619258    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grlns\" (UniqueName: \"kubernetes.io/projected/dfb8c540-571e-4f74-99c0-a6dc852fe900-kube-api-access-grlns\") pod \"payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.619434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.619297    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfb8c540-571e-4f74-99c0-a6dc852fe900-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.650580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.650559    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-554f46\""
Oct 30 21:35:41.652323 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.652302    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"]
Oct 30 21:35:41.701388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.701332    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"]
Oct 30 21:35:41.705334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.705310    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:41.720984 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.720953    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grlns\" (UniqueName: \"kubernetes.io/projected/dfb8c540-571e-4f74-99c0-a6dc852fe900-kube-api-access-grlns\") pod \"payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.721120 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.721008    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfb8c540-571e-4f74-99c0-a6dc852fe900-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.721835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.721816    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfb8c540-571e-4f74-99c0-a6dc852fe900-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.746674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.746647    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"]
Oct 30 21:35:41.779406 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.779383    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grlns\" (UniqueName: \"kubernetes.io/projected/dfb8c540-571e-4f74-99c0-a6dc852fe900-kube-api-access-grlns\") pod \"payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.822262 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.822209    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np8wf\" (UniqueName: \"kubernetes.io/projected/f49b2817-d5d6-438f-819a-58a8fbc69e3d-kube-api-access-np8wf\") pod \"payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:41.822400 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.822377    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49b2817-d5d6-438f-819a-58a8fbc69e3d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:41.868153 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.868128    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:41.925060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.925027    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49b2817-d5d6-438f-819a-58a8fbc69e3d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:41.925199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.925177    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np8wf\" (UniqueName: \"kubernetes.io/projected/f49b2817-d5d6-438f-819a-58a8fbc69e3d-kube-api-access-np8wf\") pod \"payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:41.926893 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.926854    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49b2817-d5d6-438f-819a-58a8fbc69e3d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:41.983730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:41.983705    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np8wf\" (UniqueName: \"kubernetes.io/projected/f49b2817-d5d6-438f-819a-58a8fbc69e3d-kube-api-access-np8wf\") pod \"payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:42.021735 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:42.021705    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:42.234300 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:42.234276    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"]
Oct 30 21:35:42.236699 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:42.236640    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb8c540_571e_4f74_99c0_a6dc852fe900.slice/crio-f286686faccbec9eb0c3a86b99f41e5702cb9f44283d435c41e566d7cbdcc40d WatchSource:0}: Error finding container f286686faccbec9eb0c3a86b99f41e5702cb9f44283d435c41e566d7cbdcc40d: Status 404 returned error can't find the container with id f286686faccbec9eb0c3a86b99f41e5702cb9f44283d435c41e566d7cbdcc40d
Oct 30 21:35:42.386446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:42.386424    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"]
Oct 30 21:35:42.389153 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:42.389124    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf49b2817_d5d6_438f_819a_58a8fbc69e3d.slice/crio-3231fd55f4090a4852f4185c895d6f7c3183837d8980177b75d229d8dd23df76 WatchSource:0}: Error finding container 3231fd55f4090a4852f4185c895d6f7c3183837d8980177b75d229d8dd23df76: Status 404 returned error can't find the container with id 3231fd55f4090a4852f4185c895d6f7c3183837d8980177b75d229d8dd23df76
Oct 30 21:35:42.428938 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:42.428910    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" event={"ID":"dfb8c540-571e-4f74-99c0-a6dc852fe900","Type":"ContainerStarted","Data":"f286686faccbec9eb0c3a86b99f41e5702cb9f44283d435c41e566d7cbdcc40d"}
Oct 30 21:35:42.430122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:42.430101    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" event={"ID":"f49b2817-d5d6-438f-819a-58a8fbc69e3d","Type":"ContainerStarted","Data":"3231fd55f4090a4852f4185c895d6f7c3183837d8980177b75d229d8dd23df76"}
Oct 30 21:35:45.447648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:45.447620    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" event={"ID":"dfb8c540-571e-4f74-99c0-a6dc852fe900","Type":"ContainerStarted","Data":"874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64"}
Oct 30 21:35:45.448031 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:45.447759    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:45.560296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:45.560213    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" podStartSLOduration=1.492055207 podStartE2EDuration="4.560201091s" totalImagesPullingTime="3.068145884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:45.558897537 +0000 UTC m=+1280.846620084" watchObservedRunningTime="2026-10-30 21:35:45.560201091 +0000 UTC m=+1280.847923637"
Oct 30 21:35:46.336989 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:35:46.336949    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:35:46.454123 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.454090    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" event={"ID":"f49b2817-d5d6-438f-819a-58a8fbc69e3d","Type":"ContainerStarted","Data":"98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf"}
Oct 30 21:35:46.612334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.612224    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" podStartSLOduration=2.594934255 podStartE2EDuration="5.612209967s" totalImagesPullingTime="3.017275712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:46.56592588 +0000 UTC m=+1281.853648429" watchObservedRunningTime="2026-10-30 21:35:46.612209967 +0000 UTC m=+1281.899932538"
Oct 30 21:35:46.689970 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.689939    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"]
Oct 30 21:35:46.693951 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.693930    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:46.743292 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.743265    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-71aab5\""
Oct 30 21:35:46.743452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.743418    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-71aab5-dockercfg-qgr8s\""
Oct 30 21:35:46.748665 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.748639    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"]
Oct 30 21:35:46.783389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.783330    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpz92\" (UniqueName: \"kubernetes.io/projected/5b8250db-6367-4c18-8898-c0561e4364e1-kube-api-access-jpz92\") pod \"payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:46.783577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.783552    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8250db-6367-4c18-8898-c0561e4364e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:46.816154 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.816128    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"]
Oct 30 21:35:46.830122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.830088    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:46.876032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.875962    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"]
Oct 30 21:35:46.884721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.884692    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8250db-6367-4c18-8898-c0561e4364e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:46.884879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.884760    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae986a90-e7b8-4b48-a0ae-aa39e6524693-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:46.884879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.884791    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsg62\" (UniqueName: \"kubernetes.io/projected/ae986a90-e7b8-4b48-a0ae-aa39e6524693-kube-api-access-dsg62\") pod \"payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:46.884979 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.884930    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpz92\" (UniqueName: \"kubernetes.io/projected/5b8250db-6367-4c18-8898-c0561e4364e1-kube-api-access-jpz92\") pod \"payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:46.885606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.885583    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8250db-6367-4c18-8898-c0561e4364e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:46.945992 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.945950    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpz92\" (UniqueName: \"kubernetes.io/projected/5b8250db-6367-4c18-8898-c0561e4364e1-kube-api-access-jpz92\") pod \"payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:46.986267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.986229    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae986a90-e7b8-4b48-a0ae-aa39e6524693-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:46.986267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.986267    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsg62\" (UniqueName: \"kubernetes.io/projected/ae986a90-e7b8-4b48-a0ae-aa39e6524693-kube-api-access-dsg62\") pod \"payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:46.987056 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:46.987036    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae986a90-e7b8-4b48-a0ae-aa39e6524693-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:47.011370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.007527    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:47.045044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.044794    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsg62\" (UniqueName: \"kubernetes.io/projected/ae986a90-e7b8-4b48-a0ae-aa39e6524693-kube-api-access-dsg62\") pod \"payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:47.160114 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.160036    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:47.384694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.384577    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"]
Oct 30 21:35:47.387224 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:47.387196    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8250db_6367_4c18_8898_c0561e4364e1.slice/crio-70fa451d4e964dab98060b66d321e276c4a03e70e8d1959587eb2caf43bf847e WatchSource:0}: Error finding container 70fa451d4e964dab98060b66d321e276c4a03e70e8d1959587eb2caf43bf847e: Status 404 returned error can't find the container with id 70fa451d4e964dab98060b66d321e276c4a03e70e8d1959587eb2caf43bf847e
Oct 30 21:35:47.460034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.460005    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" event={"ID":"5b8250db-6367-4c18-8898-c0561e4364e1","Type":"ContainerStarted","Data":"70fa451d4e964dab98060b66d321e276c4a03e70e8d1959587eb2caf43bf847e"}
Oct 30 21:35:47.460309 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.460147    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:47.550679 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.550648    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"]
Oct 30 21:35:47.554536 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:47.554501    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae986a90_e7b8_4b48_a0ae_aa39e6524693.slice/crio-6e81e42bcf1503f5abd741ecaa64e7d2f86da6260d5b4a33f521631f3c121d74 WatchSource:0}: Error finding container 6e81e42bcf1503f5abd741ecaa64e7d2f86da6260d5b4a33f521631f3c121d74: Status 404 returned error can't find the container with id 6e81e42bcf1503f5abd741ecaa64e7d2f86da6260d5b4a33f521631f3c121d74
Oct 30 21:35:47.626471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.626439    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"]
Oct 30 21:35:47.631444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.631420    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.680906 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.680850    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-19dd4f-serving-cert\""
Oct 30 21:35:47.687318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.687295    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"]
Oct 30 21:35:47.722987 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.722948    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/122cf522-ccbf-409b-9956-baf8a6ff103a-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.723179 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.723002    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.723179 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.723111    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-metrics-certs\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.723258 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.723187    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skw7x\" (UniqueName: \"kubernetes.io/projected/122cf522-ccbf-409b-9956-baf8a6ff103a-kube-api-access-skw7x\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.823988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.823947    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/122cf522-ccbf-409b-9956-baf8a6ff103a-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.823988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.823993    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.824244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.824027    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-metrics-certs\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.824244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.824075    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skw7x\" (UniqueName: \"kubernetes.io/projected/122cf522-ccbf-409b-9956-baf8a6ff103a-kube-api-access-skw7x\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.824244 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:35:47.824142    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-serving-cert: secret "maas-api-e2e-worker-w1-19dd4f-serving-cert" not found
Oct 30 21:35:47.824244 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:35:47.824241    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls podName:122cf522-ccbf-409b-9956-baf8a6ff103a nodeName:}" failed. No retries permitted until 2026-10-30 21:35:48.324205088 +0000 UTC m=+1283.611927629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls") pod "maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4" (UID: "122cf522-ccbf-409b-9956-baf8a6ff103a") : secret "maas-api-e2e-worker-w1-19dd4f-serving-cert" not found
Oct 30 21:35:47.824769 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.824742    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/122cf522-ccbf-409b-9956-baf8a6ff103a-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.826492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.826471    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-metrics-certs\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:47.884451 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:47.884417    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skw7x\" (UniqueName: \"kubernetes.io/projected/122cf522-ccbf-409b-9956-baf8a6ff103a-kube-api-access-skw7x\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:48.329925 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.329891    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:48.332314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.332293    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls\") pod \"maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:48.467141 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.467105    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" event={"ID":"5b8250db-6367-4c18-8898-c0561e4364e1","Type":"ContainerStarted","Data":"e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef"}
Oct 30 21:35:48.467659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.467235    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:48.468917 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.468889    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" event={"ID":"ae986a90-e7b8-4b48-a0ae-aa39e6524693","Type":"ContainerStarted","Data":"49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684"}
Oct 30 21:35:48.469061 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.468929    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" event={"ID":"ae986a90-e7b8-4b48-a0ae-aa39e6524693","Type":"ContainerStarted","Data":"6e81e42bcf1503f5abd741ecaa64e7d2f86da6260d5b4a33f521631f3c121d74"}
Oct 30 21:35:48.469061 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.469046    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:48.553961 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.553924    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:48.579825 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.579775    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" podStartSLOduration=2.579760079 podStartE2EDuration="2.579760079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:48.578912783 +0000 UTC m=+1283.866635331" watchObservedRunningTime="2026-10-30 21:35:48.579760079 +0000 UTC m=+1283.867482624"
Oct 30 21:35:48.946363 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.946302    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" podStartSLOduration=2.946288144 podStartE2EDuration="2.946288144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:48.712994563 +0000 UTC m=+1284.000717111" watchObservedRunningTime="2026-10-30 21:35:48.946288144 +0000 UTC m=+1284.234010690"
Oct 30 21:35:48.946474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:48.946460    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"]
Oct 30 21:35:48.948782 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:48.948754    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod122cf522_ccbf_409b_9956_baf8a6ff103a.slice/crio-9a24ca64d03456e5af337d002c6f485ac0b38f60fdc8a3dd48cc0b2f262a58a8 WatchSource:0}: Error finding container 9a24ca64d03456e5af337d002c6f485ac0b38f60fdc8a3dd48cc0b2f262a58a8: Status 404 returned error can't find the container with id 9a24ca64d03456e5af337d002c6f485ac0b38f60fdc8a3dd48cc0b2f262a58a8
Oct 30 21:35:49.475663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:49.475625    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4" event={"ID":"122cf522-ccbf-409b-9956-baf8a6ff103a","Type":"ContainerStarted","Data":"9a24ca64d03456e5af337d002c6f485ac0b38f60fdc8a3dd48cc0b2f262a58a8"}
Oct 30 21:35:50.486401 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:50.486356    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4" event={"ID":"122cf522-ccbf-409b-9956-baf8a6ff103a","Type":"ContainerStarted","Data":"c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604"}
Oct 30 21:35:50.487290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:50.487264    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:50.614992 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:50.614930    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4" podStartSLOduration=2.7498708929999998 podStartE2EDuration="3.614911474s" totalImagesPullingTime="865.040581ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:50.608701287 +0000 UTC m=+1285.896423845" watchObservedRunningTime="2026-10-30 21:35:50.614911474 +0000 UTC m=+1285.902634023"
Oct 30 21:35:51.927297 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:51.927264    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"]
Oct 30 21:35:51.931279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:51.931258    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.007366 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.005612    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-991335\""
Oct 30 21:35:52.057370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.056081    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"]
Oct 30 21:35:52.057370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.056261    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-991335-dockercfg-zf9fj\""
Oct 30 21:35:52.073318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.073284    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df85e97b-dc4d-4b76-8977-0c721046960d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.073318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.073323    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6sp7\" (UniqueName: \"kubernetes.io/projected/df85e97b-dc4d-4b76-8977-0c721046960d-kube-api-access-h6sp7\") pod \"payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.174458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.174426    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df85e97b-dc4d-4b76-8977-0c721046960d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.174606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.174466    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6sp7\" (UniqueName: \"kubernetes.io/projected/df85e97b-dc4d-4b76-8977-0c721046960d-kube-api-access-h6sp7\") pod \"payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.175089 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.175072    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df85e97b-dc4d-4b76-8977-0c721046960d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.232809 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.232778    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6sp7\" (UniqueName: \"kubernetes.io/projected/df85e97b-dc4d-4b76-8977-0c721046960d-kube-api-access-h6sp7\") pod \"payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.244395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.244366    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:52.640177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:52.640150    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"]
Oct 30 21:35:52.642790 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:52.642754    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf85e97b_dc4d_4b76_8977_0c721046960d.slice/crio-64841bd4e367baa93ba9babfc8c88b2ad13430b3c457c5fdcfdadbec7ba392cd WatchSource:0}: Error finding container 64841bd4e367baa93ba9babfc8c88b2ad13430b3c457c5fdcfdadbec7ba392cd: Status 404 returned error can't find the container with id 64841bd4e367baa93ba9babfc8c88b2ad13430b3c457c5fdcfdadbec7ba392cd
Oct 30 21:35:53.527865 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:53.527822    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" event={"ID":"df85e97b-dc4d-4b76-8977-0c721046960d","Type":"ContainerStarted","Data":"e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032"}
Oct 30 21:35:53.528383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:53.527893    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" event={"ID":"df85e97b-dc4d-4b76-8977-0c721046960d","Type":"ContainerStarted","Data":"64841bd4e367baa93ba9babfc8c88b2ad13430b3c457c5fdcfdadbec7ba392cd"}
Oct 30 21:35:53.528383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:53.527928    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:35:53.638782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:53.638725    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" podStartSLOduration=2.63871037 podStartE2EDuration="2.63871037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:53.638079973 +0000 UTC m=+1288.925802521" watchObservedRunningTime="2026-10-30 21:35:53.63871037 +0000 UTC m=+1288.926432916"
Oct 30 21:35:56.457616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.457584    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:35:56.522046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.522016    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 21:35:56.546647 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.546621    5817 generic.go:356] "Generic (PLEG): container finished" podID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerID="b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb" exitCode=0
Oct 30 21:35:56.546787 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.546678    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" event={"ID":"f7f80f7d-26be-43b8-b355-9c7e8a9924d7","Type":"ContainerDied","Data":"b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb"}
Oct 30 21:35:56.547013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.547001    5817 scope.go:122] "RemoveContainer" containerID="b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb"
Oct 30 21:35:56.548418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.548390    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:56.548498 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.548470    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:56.548535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:56.548506    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:35:57.015685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.015649    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"]
Oct 30 21:35:57.020855 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.020784    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.029524 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.029501    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67beb8a-dc3d-47e0-a876-6d6197f22003-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.029651 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.029618    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxh7p\" (UniqueName: \"kubernetes.io/projected/c67beb8a-dc3d-47e0-a876-6d6197f22003-kube-api-access-bxh7p\") pod \"payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.070226 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.070205    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"]
Oct 30 21:35:57.070733 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.070711    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-19dd4f\""
Oct 30 21:35:57.071049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.071030    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-19dd4f-dockercfg-zrg6q\""
Oct 30 21:35:57.140425 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.140185    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxh7p\" (UniqueName: \"kubernetes.io/projected/c67beb8a-dc3d-47e0-a876-6d6197f22003-kube-api-access-bxh7p\") pod \"payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.140425 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.140319    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67beb8a-dc3d-47e0-a876-6d6197f22003-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.141536 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.141502    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67beb8a-dc3d-47e0-a876-6d6197f22003-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.144854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.144828    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"]
Oct 30 21:35:57.149002 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.148980    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.218548 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.218069    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxh7p\" (UniqueName: \"kubernetes.io/projected/c67beb8a-dc3d-47e0-a876-6d6197f22003-kube-api-access-bxh7p\") pod \"payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.218738 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.218651    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"]
Oct 30 21:35:57.241447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.241414    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fce027b-4ba9-4fb6-a01f-debcc062df5c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.241597 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.241474    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np44t\" (UniqueName: \"kubernetes.io/projected/7fce027b-4ba9-4fb6-a01f-debcc062df5c-kube-api-access-np44t\") pod \"payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.334424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.333557    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:57.343360 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.343301    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fce027b-4ba9-4fb6-a01f-debcc062df5c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.343452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.343407    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np44t\" (UniqueName: \"kubernetes.io/projected/7fce027b-4ba9-4fb6-a01f-debcc062df5c-kube-api-access-np44t\") pod \"payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.343943 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.343923    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fce027b-4ba9-4fb6-a01f-debcc062df5c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.460829 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.460167    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np44t\" (UniqueName: \"kubernetes.io/projected/7fce027b-4ba9-4fb6-a01f-debcc062df5c-kube-api-access-np44t\") pod \"payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.463409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.463372    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:57.556402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.556367    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" event={"ID":"f7f80f7d-26be-43b8-b355-9c7e8a9924d7","Type":"ContainerStarted","Data":"e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96"}
Oct 30 21:35:57.642453 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.638277    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:35:57.642744 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.642711    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:35:57.642818 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.642769    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:35:57.746646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.746604    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"]
Oct 30 21:35:57.757291 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:57.757257    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67beb8a_dc3d_47e0_a876_6d6197f22003.slice/crio-42cc1eec12272edda559afa3a4cdd3537dac90dce6db66a0a74ef1eafc736115 WatchSource:0}: Error finding container 42cc1eec12272edda559afa3a4cdd3537dac90dce6db66a0a74ef1eafc736115: Status 404 returned error can't find the container with id 42cc1eec12272edda559afa3a4cdd3537dac90dce6db66a0a74ef1eafc736115
Oct 30 21:35:57.884610 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.884585    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"]
Oct 30 21:35:57.886698 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:35:57.886673    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fce027b_4ba9_4fb6_a01f_debcc062df5c.slice/crio-2644f52760cb047dd50666ce69704338600433e5ffb7ea41aa01553c83cd2f11 WatchSource:0}: Error finding container 2644f52760cb047dd50666ce69704338600433e5ffb7ea41aa01553c83cd2f11: Status 404 returned error can't find the container with id 2644f52760cb047dd50666ce69704338600433e5ffb7ea41aa01553c83cd2f11
Oct 30 21:35:57.944843 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.944804    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"]
Oct 30 21:35:57.951290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.951268    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:57.969886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.969860    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-metrics-certs\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:57.970004 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.969927    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:57.970004 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.969980    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdkb8\" (UniqueName: \"kubernetes.io/projected/5930a3d6-c4e4-4518-bee4-de0e1340a990-kube-api-access-xdkb8\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:57.970115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.970050    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5930a3d6-c4e4-4518-bee4-de0e1340a990-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:57.980960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:57.980919    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"]
Oct 30 21:35:58.010377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.003141    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-ed76b8-serving-cert\""
Oct 30 21:35:58.071378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.070898    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-metrics-certs\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.071378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.070949    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.071378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.070981    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdkb8\" (UniqueName: \"kubernetes.io/projected/5930a3d6-c4e4-4518-bee4-de0e1340a990-kube-api-access-xdkb8\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.071378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.071025    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5930a3d6-c4e4-4518-bee4-de0e1340a990-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.071696 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:35:58.071410    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-serving-cert: secret "maas-api-e2e-worker-w2-ed76b8-serving-cert" not found
Oct 30 21:35:58.071696 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:35:58.071476    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls podName:5930a3d6-c4e4-4518-bee4-de0e1340a990 nodeName:}" failed. No retries permitted until 2026-10-30 21:35:58.571454653 +0000 UTC m=+1293.859177189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls") pod "maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v" (UID: "5930a3d6-c4e4-4518-bee4-de0e1340a990") : secret "maas-api-e2e-worker-w2-ed76b8-serving-cert" not found
Oct 30 21:35:58.071696 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.071666    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5930a3d6-c4e4-4518-bee4-de0e1340a990-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.073597 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.073567    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-metrics-certs\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.142419 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.140973    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdkb8\" (UniqueName: \"kubernetes.io/projected/5930a3d6-c4e4-4518-bee4-de0e1340a990-kube-api-access-xdkb8\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.471640 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.471613    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:35:58.563926 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.563860    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" event={"ID":"7fce027b-4ba9-4fb6-a01f-debcc062df5c","Type":"ContainerStarted","Data":"333981d416721e2aadaacedea05ae5a810c6a3a7c41734c9de8d87b6d3ea4035"}
Oct 30 21:35:58.563926 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.563920    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" event={"ID":"7fce027b-4ba9-4fb6-a01f-debcc062df5c","Type":"ContainerStarted","Data":"2644f52760cb047dd50666ce69704338600433e5ffb7ea41aa01553c83cd2f11"}
Oct 30 21:35:58.564179 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.563946    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:35:58.565447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.565332    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" event={"ID":"c67beb8a-dc3d-47e0-a876-6d6197f22003","Type":"ContainerStarted","Data":"0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262"}
Oct 30 21:35:58.565447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.565399    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" event={"ID":"c67beb8a-dc3d-47e0-a876-6d6197f22003","Type":"ContainerStarted","Data":"42cc1eec12272edda559afa3a4cdd3537dac90dce6db66a0a74ef1eafc736115"}
Oct 30 21:35:58.565447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.565424    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:35:58.576178 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.576152    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.578784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.578763    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.581782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.581620    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:35:58.646681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.646562    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:35:58.646681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.646644    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:35:58.876301 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.876235    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" podStartSLOduration=2.876218498 podStartE2EDuration="2.876218498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:58.737931011 +0000 UTC m=+1294.025653559" watchObservedRunningTime="2026-10-30 21:35:58.876218498 +0000 UTC m=+1294.163941044"
Oct 30 21:35:58.876498 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:58.876404    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" podStartSLOduration=1.87639568 podStartE2EDuration="1.87639568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:58.874267185 +0000 UTC m=+1294.161989732" watchObservedRunningTime="2026-10-30 21:35:58.87639568 +0000 UTC m=+1294.164118228"
Oct 30 21:35:59.087541 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:59.087518    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"]
Oct 30 21:35:59.113526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:59.113490    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:35:59.478446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:59.478144    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 21:35:59.478832 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:59.478520    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 21:35:59.572268 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:59.572197    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v" event={"ID":"5930a3d6-c4e4-4518-bee4-de0e1340a990","Type":"ContainerStarted","Data":"6a95580e3a5e72b534ab2ea2df0a71f1643d72d4d5cc3ddaec675db0382d3b9c"}
Oct 30 21:35:59.634773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:59.634728    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:35:59.634970 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:35:59.634795    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:36:00.360490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:00.360404    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:36:00.627412 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:00.626201    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v" event={"ID":"5930a3d6-c4e4-4518-bee4-de0e1340a990","Type":"ContainerStarted","Data":"be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b"}
Oct 30 21:36:00.627412 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:00.627220    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:36:00.636881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:00.636443    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:36:00.636881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:00.636499    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:36:01.633825 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:01.633789    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:36:01.634566 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:01.634519    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:36:02.633466 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:02.633423    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:36:02.633628 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:02.633485    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:36:03.638570 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:03.638536    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:36:03.639650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:03.639628    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 21:36:03.805246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:03.805196    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v" podStartSLOduration=5.847153896 podStartE2EDuration="6.805183229s" totalImagesPullingTime="958.029333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:00.741611153 +0000 UTC m=+1296.029333702" watchObservedRunningTime="2026-10-30 21:36:03.805183229 +0000 UTC m=+1299.092905773"
Oct 30 21:36:04.536152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:04.536123    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 21:36:05.671039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:05.670937    5817 generic.go:356] "Generic (PLEG): container finished" podID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerID="41e1149c1a6943960b443842ebbd9b9838247375d37f8e340225d68296b25e03" exitCode=0
Oct 30 21:36:05.671039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:05.671013    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" event={"ID":"9d54ac74-962b-4549-8c9b-a2c98f4d3251","Type":"ContainerDied","Data":"41e1149c1a6943960b443842ebbd9b9838247375d37f8e340225d68296b25e03"}
Oct 30 21:36:05.671535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:05.671402    5817 scope.go:122] "RemoveContainer" containerID="41e1149c1a6943960b443842ebbd9b9838247375d37f8e340225d68296b25e03"
Oct 30 21:36:05.672593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:05.672548    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:05.672704 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:05.672643    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:05.672704 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:05.672689    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:06.678272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:06.678237    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" event={"ID":"9d54ac74-962b-4549-8c9b-a2c98f4d3251","Type":"ContainerStarted","Data":"b58345370de7281cd256639a4dbe45a03dae272c2067b364178b17cd5d1929a1"}
Oct 30 21:36:07.525195 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:07.525157    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:36:07.525418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:07.525396    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:36:07.525508 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:07.525467    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:36:07.736370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:07.734789    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 21:36:08.525145 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:08.525110    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:36:08.525318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:08.525161    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:36:09.524885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:09.524850    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:36:09.525269 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:09.524898    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:36:09.575649 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:09.575622    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 21:36:09.575932 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:09.575906    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 21:36:10.525138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:10.525105    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:36:10.525543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:10.525155    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:36:11.525635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:11.525595    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 21:36:11.526105 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:11.525646    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 21:36:12.529372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:12.529318    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:36:12.529919 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:12.529900    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:36:12.530499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:12.530481    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 21:36:16.337509 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:36:16.337473    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:36:27.876208 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:27.876171    5817 generic.go:356] "Generic (PLEG): container finished" podID="ff45733c-4a30-43c1-b509-18792c82ec04" containerID="38019071fc0d63dce06612171d26127afcc6b1c2595983311f25c9d7ea9ace40" exitCode=0
Oct 30 21:36:27.876754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:27.876213    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" event={"ID":"ff45733c-4a30-43c1-b509-18792c82ec04","Type":"ContainerDied","Data":"38019071fc0d63dce06612171d26127afcc6b1c2595983311f25c9d7ea9ace40"}
Oct 30 21:36:27.876754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:27.876658    5817 scope.go:122] "RemoveContainer" containerID="38019071fc0d63dce06612171d26127afcc6b1c2595983311f25c9d7ea9ace40"
Oct 30 21:36:27.877965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:27.877931    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:27.878060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:27.878002    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:27.878060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:27.878054    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:28.883244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:28.883206    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" event={"ID":"ff45733c-4a30-43c1-b509-18792c82ec04","Type":"ContainerStarted","Data":"b28c534845a777a744a5c8e6060c6d53dd37c1f49cf617c3d28c78ae599b72f0"}
Oct 30 21:36:29.815844 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:29.815791    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:36:29.816089 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:29.815860    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:36:29.816212 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:29.816187    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:36:30.816859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:30.816816    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:36:30.817448 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:30.817413    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:36:31.814841 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:31.814800    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:36:31.815113 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:31.815086    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:36:32.372471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:32.372438    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:36:32.817371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:32.816730    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:36:32.817371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:32.816812    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:36:33.816777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:33.816732    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 21:36:33.817297 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:33.816799    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 21:36:34.818953 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:34.818923    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:36:34.819972 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:34.819948    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 21:36:39.997607 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:39.997574    5817 generic.go:356] "Generic (PLEG): container finished" podID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerID="7bb86c03431c663a0c69b12e7c4231250d7959738ac5a2b863be2177a8b736e1" exitCode=0
Oct 30 21:36:39.998023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:39.997647    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" event={"ID":"2c8f0730-9523-4719-a5d9-f3aee98bb8b5","Type":"ContainerDied","Data":"7bb86c03431c663a0c69b12e7c4231250d7959738ac5a2b863be2177a8b736e1"}
Oct 30 21:36:39.998064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:39.998026    5817 scope.go:122] "RemoveContainer" containerID="7bb86c03431c663a0c69b12e7c4231250d7959738ac5a2b863be2177a8b736e1"
Oct 30 21:36:39.999384 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:39.999330    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:39.999444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:39.999430    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:39.999479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:39.999469    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:36:41.019267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:41.019233    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" event={"ID":"2c8f0730-9523-4719-a5d9-f3aee98bb8b5","Type":"ContainerStarted","Data":"b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2"}
Oct 30 21:36:41.230468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:41.230430    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:36:41.230750 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:41.230721    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:36:41.230809 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:41.230765    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:36:42.230640 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:42.230604    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:36:42.231021 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:42.230660    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:36:43.229849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:43.229816    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:36:43.230031 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:43.229869    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:36:44.229886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:44.229849    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:36:44.230301 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:44.229903    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:36:45.230468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:45.230430    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:36:45.230953 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:45.230494    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 21:36:46.233925 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:46.233892    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:36:46.234296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:46.234177    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:36:46.234895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:36:46.234877    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:36:46.337683 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:36:46.337652    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:37:16.336920 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:37:16.336847    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:37:29.429947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.429912    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"]
Oct 30 21:37:29.432894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.432870    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.508590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.508555    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tmp-dir\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.508754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.508612    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-home\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.508754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.508689    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-dshm\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.508754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.508739    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tls-certs\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.508883 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.508841    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-model-cache\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.508883 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.508863    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65bf4\" (UniqueName: \"kubernetes.io/projected/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-kube-api-access-65bf4\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.529206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.529180    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-991335\"/\"default-dockercfg-9scmz\""
Oct 30 21:37:29.529313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.529231    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-991335\"/\"openshift-service-ca.crt\""
Oct 30 21:37:29.529313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.529241    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-991335\"/\"simulator-w5-991335-kserve-self-signed-certs\""
Oct 30 21:37:29.529313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.529243    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-991335\"/\"kube-root-ca.crt\""
Oct 30 21:37:29.531331 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.531314    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"]
Oct 30 21:37:29.609480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609450    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tmp-dir\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.609648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609586    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-home\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.609648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609612    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-dshm\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.609648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609626    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tls-certs\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.609766 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609672    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-model-cache\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.609766 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609695    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65bf4\" (UniqueName: \"kubernetes.io/projected/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-kube-api-access-65bf4\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.609864 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609834    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tmp-dir\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.609978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.609954    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-home\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.610044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.610016    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-model-cache\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.611802 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.611785    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-dshm\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.612129 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.612113    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tls-certs\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.668067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.668039    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65bf4\" (UniqueName: \"kubernetes.io/projected/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-kube-api-access-65bf4\") pod \"simulator-w5-991335-kserve-7d6df54b78-dbkm6\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") " pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:29.743863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:29.743837    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:30.115006 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:30.114980    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"]
Oct 30 21:37:30.117450 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:37:30.117423    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57ecfea3_2e0a_4ead_b994_6bc7aca1aa52.slice/crio-d93a08fc55f28249599227f43aaff574cc1b74e10565c37465143cef0c1c2f60 WatchSource:0}: Error finding container d93a08fc55f28249599227f43aaff574cc1b74e10565c37465143cef0c1c2f60: Status 404 returned error can't find the container with id d93a08fc55f28249599227f43aaff574cc1b74e10565c37465143cef0c1c2f60
Oct 30 21:37:30.305701 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:30.305671    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" event={"ID":"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52","Type":"ContainerStarted","Data":"e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328"}
Oct 30 21:37:30.305701 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:30.305708    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" event={"ID":"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52","Type":"ContainerStarted","Data":"d93a08fc55f28249599227f43aaff574cc1b74e10565c37465143cef0c1c2f60"}
Oct 30 21:37:30.420803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:30.420699    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podStartSLOduration=1.420685429 podStartE2EDuration="1.420685429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:30.418560765 +0000 UTC m=+1385.706283310" watchObservedRunningTime="2026-10-30 21:37:30.420685429 +0000 UTC m=+1385.708407974"
Oct 30 21:37:30.743941 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:30.743904    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:39.744245 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:39.744210    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:39.757088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:39.757059    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:40.376525 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:40.376494    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:37:46.337339 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:37:46.337302    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:37:58.779885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.779849    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"]
Oct 30 21:37:58.782895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.782866    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:58.879190 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.879155    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-71aab5\"/\"kube-root-ca.crt\""
Oct 30 21:37:58.879190 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.879160    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-71aab5\"/\"openshift-service-ca.crt\""
Oct 30 21:37:58.879474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.879267    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-71aab5\"/\"default-dockercfg-25t4z\""
Oct 30 21:37:58.879474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.879373    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-71aab5\"/\"simulator-w3-71aab5-kserve-self-signed-certs\""
Oct 30 21:37:58.881408 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.881388    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"]
Oct 30 21:37:58.910672 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.910634    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-home\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:58.910836 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.910699    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-dshm\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:58.910836 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.910783    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh4x7\" (UniqueName: \"kubernetes.io/projected/6c05da20-a98a-4b76-9db6-44d92303fe12-kube-api-access-gh4x7\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:58.910921 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.910842    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c05da20-a98a-4b76-9db6-44d92303fe12-tls-certs\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:58.910962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.910941    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-model-cache\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:58.911025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:58.911013    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-tmp-dir\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.012329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.012284    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-tmp-dir\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.012329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.012339    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-home\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.012580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.012409    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-dshm\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.012580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.012436    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh4x7\" (UniqueName: \"kubernetes.io/projected/6c05da20-a98a-4b76-9db6-44d92303fe12-kube-api-access-gh4x7\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.012580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.012493    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c05da20-a98a-4b76-9db6-44d92303fe12-tls-certs\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.012580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.012540    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-model-cache\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.018612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.013177    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-home\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.018612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.013206    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-tmp-dir\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.018612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.013410    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-model-cache\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.018612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.015542    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-dshm\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.022370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.022329    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c05da20-a98a-4b76-9db6-44d92303fe12-tls-certs\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.070325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.070258    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh4x7\" (UniqueName: \"kubernetes.io/projected/6c05da20-a98a-4b76-9db6-44d92303fe12-kube-api-access-gh4x7\") pod \"simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") " pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.094271 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.094246    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:37:59.462049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:37:59.462015    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"]
Oct 30 21:37:59.464442 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:37:59.464410    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c05da20_a98a_4b76_9db6_44d92303fe12.slice/crio-1a2416229dbf322dfdf154b37be8dda607e1840820eb2e4cc28be216f2733a4f WatchSource:0}: Error finding container 1a2416229dbf322dfdf154b37be8dda607e1840820eb2e4cc28be216f2733a4f: Status 404 returned error can't find the container with id 1a2416229dbf322dfdf154b37be8dda607e1840820eb2e4cc28be216f2733a4f
Oct 30 21:38:00.477180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:00.477143    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" event={"ID":"6c05da20-a98a-4b76-9db6-44d92303fe12","Type":"ContainerStarted","Data":"c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d"}
Oct 30 21:38:00.477180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:00.477187    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" event={"ID":"6c05da20-a98a-4b76-9db6-44d92303fe12","Type":"ContainerStarted","Data":"1a2416229dbf322dfdf154b37be8dda607e1840820eb2e4cc28be216f2733a4f"}
Oct 30 21:38:00.589835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:00.589782    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podStartSLOduration=2.589766553 podStartE2EDuration="2.589766553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:00.588394984 +0000 UTC m=+1415.876117530" watchObservedRunningTime="2026-10-30 21:38:00.589766553 +0000 UTC m=+1415.877489093"
Oct 30 21:38:01.095001 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:01.094958    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:38:09.094928 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.094889    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:38:09.106890 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.106865    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:38:09.539454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.539424    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:38:09.706768 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.706735    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"]
Oct 30 21:38:09.709631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.709610    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.724764 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.724742    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-dshm\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.724881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.724774    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-home\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.724922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.724876    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f3244d-34b1-4025-8777-5db36836e0c8-tls-certs\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.724922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.724903    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-tmp-dir\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.725018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.724999    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnf97\" (UniqueName: \"kubernetes.io/projected/57f3244d-34b1-4025-8777-5db36836e0c8-kube-api-access-fnf97\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.725051 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.725033    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-model-cache\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.759460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.759434    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-19dd4f\"/\"openshift-service-ca.crt\""
Oct 30 21:38:09.759460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.759432    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-19dd4f\"/\"kube-root-ca.crt\""
Oct 30 21:38:09.759615 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.759457    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-19dd4f\"/\"premium-w1-19dd4f-kserve-self-signed-certs\""
Oct 30 21:38:09.759615 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.759499    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-19dd4f\"/\"default-dockercfg-w6f95\""
Oct 30 21:38:09.765788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.765767    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"]
Oct 30 21:38:09.826010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.825924    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-tmp-dir\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826013    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnf97\" (UniqueName: \"kubernetes.io/projected/57f3244d-34b1-4025-8777-5db36836e0c8-kube-api-access-fnf97\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826048    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-model-cache\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826135    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-dshm\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826270 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826165    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-home\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826270 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826261    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f3244d-34b1-4025-8777-5db36836e0c8-tls-certs\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826386    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-tmp-dir\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826497 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826477    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-model-cache\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.826633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.826616    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-home\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.828327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.828296    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-dshm\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.828591 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.828572    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f3244d-34b1-4025-8777-5db36836e0c8-tls-certs\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:09.884556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:09.884522    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnf97\" (UniqueName: \"kubernetes.io/projected/57f3244d-34b1-4025-8777-5db36836e0c8-kube-api-access-fnf97\") pod \"premium-w1-19dd4f-kserve-7bdff55b77-hjjwh\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") " pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:10.021552 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:10.021513    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:10.393095 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:10.393069    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"]
Oct 30 21:38:10.395175 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:38:10.395146    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57f3244d_34b1_4025_8777_5db36836e0c8.slice/crio-7a025815b492823e488194178b5f4d80c94e3e9d9afd3119edaccb723d96b890 WatchSource:0}: Error finding container 7a025815b492823e488194178b5f4d80c94e3e9d9afd3119edaccb723d96b890: Status 404 returned error can't find the container with id 7a025815b492823e488194178b5f4d80c94e3e9d9afd3119edaccb723d96b890
Oct 30 21:38:10.535214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:10.535181    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" event={"ID":"57f3244d-34b1-4025-8777-5db36836e0c8","Type":"ContainerStarted","Data":"75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023"}
Oct 30 21:38:10.535391 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:10.535229    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" event={"ID":"57f3244d-34b1-4025-8777-5db36836e0c8","Type":"ContainerStarted","Data":"7a025815b492823e488194178b5f4d80c94e3e9d9afd3119edaccb723d96b890"}
Oct 30 21:38:10.654970 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:10.654883    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podStartSLOduration=1.654869285 podStartE2EDuration="1.654869285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:38:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:38:10.652855919 +0000 UTC m=+1425.940578465" watchObservedRunningTime="2026-10-30 21:38:10.654869285 +0000 UTC m=+1425.942591831"
Oct 30 21:38:11.021872 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:11.021841    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:16.337547 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:38:16.337502    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:38:20.022446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:20.022403    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:20.035572 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:20.035534    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:20.600223 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:38:20.600193    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:38:46.337202 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:38:46.337164    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:39:01.563516 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.563484    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"]
Oct 30 21:39:01.567033 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.567013    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.663196 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.663165    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-991335\"/\"premium-w5-991335-kserve-self-signed-certs\""
Oct 30 21:39:01.665537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.665510    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"]
Oct 30 21:39:01.717113 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.717016    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a09d72-9c45-41d6-b755-d018dc2271cb-tls-certs\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.717113 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.717084    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-model-cache\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.717413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.717179    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-tmp-dir\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.717413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.717218    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfz2z\" (UniqueName: \"kubernetes.io/projected/28a09d72-9c45-41d6-b755-d018dc2271cb-kube-api-access-bfz2z\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.717413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.717251    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-dshm\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.717413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.717295    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-home\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818323 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818249    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-dshm\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818323 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818297    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-home\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818516 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818387    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a09d72-9c45-41d6-b755-d018dc2271cb-tls-certs\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818516 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818443    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-model-cache\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818628 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818517    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-tmp-dir\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818628 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818541    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfz2z\" (UniqueName: \"kubernetes.io/projected/28a09d72-9c45-41d6-b755-d018dc2271cb-kube-api-access-bfz2z\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818773    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-home\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818833    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-model-cache\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.818936 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.818916    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-tmp-dir\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.820620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.820597    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-dshm\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.821105 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.821084    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a09d72-9c45-41d6-b755-d018dc2271cb-tls-certs\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.878522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.878495    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfz2z\" (UniqueName: \"kubernetes.io/projected/28a09d72-9c45-41d6-b755-d018dc2271cb-kube-api-access-bfz2z\") pod \"premium-w5-991335-kserve-7f6579d59f-8gldz\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") " pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:01.878653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:01.878634    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:02.244606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:02.244579    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"]
Oct 30 21:39:02.246939 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:39:02.246911    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28a09d72_9c45_41d6_b755_d018dc2271cb.slice/crio-c76105eab21d70073f83ea3d4e12467f898527e0170988e741cb661384532dfe WatchSource:0}: Error finding container c76105eab21d70073f83ea3d4e12467f898527e0170988e741cb661384532dfe: Status 404 returned error can't find the container with id c76105eab21d70073f83ea3d4e12467f898527e0170988e741cb661384532dfe
Oct 30 21:39:02.858280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:02.858242    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" event={"ID":"28a09d72-9c45-41d6-b755-d018dc2271cb","Type":"ContainerStarted","Data":"779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56"}
Oct 30 21:39:02.858765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:02.858294    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" event={"ID":"28a09d72-9c45-41d6-b755-d018dc2271cb","Type":"ContainerStarted","Data":"c76105eab21d70073f83ea3d4e12467f898527e0170988e741cb661384532dfe"}
Oct 30 21:39:02.879555 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:02.879529    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:02.972910 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:02.972848    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podStartSLOduration=1.972834723 podStartE2EDuration="1.972834723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:02.971554807 +0000 UTC m=+1478.259277352" watchObservedRunningTime="2026-10-30 21:39:02.972834723 +0000 UTC m=+1478.260557269"
Oct 30 21:39:11.878864 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:11.878830    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:11.894319 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:11.894284    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:11.929328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:11.929295    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:39:16.337741 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:39:16.337704    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:39:27.196761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.196728    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"]
Oct 30 21:39:27.200953 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.200931    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.297269 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.297237    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-71aab5\"/\"premium-w3-71aab5-kserve-self-signed-certs\""
Oct 30 21:39:27.299196 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.299173    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"]
Oct 30 21:39:27.305948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.305914    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-dshm\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.306056 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.305989    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-home\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.306102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.306083    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjkv\" (UniqueName: \"kubernetes.io/projected/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-kube-api-access-hjjkv\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.306150 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.306116    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tls-certs\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.306253 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.306232    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-model-cache\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.306327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.306310    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tmp-dir\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407289 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407256    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-home\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407525 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407424    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjkv\" (UniqueName: \"kubernetes.io/projected/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-kube-api-access-hjjkv\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407525 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407465    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tls-certs\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407629 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407532    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-model-cache\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407629 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407563    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tmp-dir\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407726 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407638    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-dshm\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407768    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-home\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.407918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407897    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-model-cache\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.408019 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.407998    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tmp-dir\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.409805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.409781    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-dshm\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.410117 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.410096    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tls-certs\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.465405 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.465378    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjkv\" (UniqueName: \"kubernetes.io/projected/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-kube-api-access-hjjkv\") pod \"premium-w3-71aab5-kserve-6d46bb4fd-5mv7s\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") " pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.513067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.513039    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:27.894974 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:27.894946    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"]
Oct 30 21:39:27.897117 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:39:27.897087    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod410f53f3_a093_4f1f_8cf6_fb7cd5eda739.slice/crio-a5b4afb5264921b82ece4f94016fc6cb0b13c33112c44f59418ce02c57f07276 WatchSource:0}: Error finding container a5b4afb5264921b82ece4f94016fc6cb0b13c33112c44f59418ce02c57f07276: Status 404 returned error can't find the container with id a5b4afb5264921b82ece4f94016fc6cb0b13c33112c44f59418ce02c57f07276
Oct 30 21:39:28.012515 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:28.012481    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" event={"ID":"410f53f3-a093-4f1f-8cf6-fb7cd5eda739","Type":"ContainerStarted","Data":"6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d"}
Oct 30 21:39:28.012515 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:28.012526    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" event={"ID":"410f53f3-a093-4f1f-8cf6-fb7cd5eda739","Type":"ContainerStarted","Data":"a5b4afb5264921b82ece4f94016fc6cb0b13c33112c44f59418ce02c57f07276"}
Oct 30 21:39:28.124681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:28.124631    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podStartSLOduration=1.124618496 podStartE2EDuration="1.124618496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:28.123840052 +0000 UTC m=+1503.411562599" watchObservedRunningTime="2026-10-30 21:39:28.124618496 +0000 UTC m=+1503.412341039"
Oct 30 21:39:28.513753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:28.513725    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:37.513546 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:37.513513    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:37.525852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:37.525826    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:38.090819 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:38.090785    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:39:46.341019 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:39:46.340977    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:39:59.743698 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.743653    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"]
Oct 30 21:39:59.748594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.748564    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.799370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.798562    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-ed76b8-openshift-default-dockercfg-47s56\""
Oct 30 21:39:59.847866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.847831    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"]
Oct 30 21:39:59.914463 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914426    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914505    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25fq2\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-kube-api-access-25fq2\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914535    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914616    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f96548-9ebe-4495-8aea-c3baf811f695-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914649    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f96548-9ebe-4495-8aea-c3baf811f695-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914672    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914843 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914775    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914843 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914812    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:39:59.914925 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:39:59.914850    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016100    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016143    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25fq2\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-kube-api-access-25fq2\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016162    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016197    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f96548-9ebe-4495-8aea-c3baf811f695-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016230    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f96548-9ebe-4495-8aea-c3baf811f695-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016263    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016375    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016458    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016534    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016635    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.016962 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016905    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.017023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.016957    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.017204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.017180    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.017316 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.017289    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f96548-9ebe-4495-8aea-c3baf811f695-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.018784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.018763    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.018905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.018822    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f96548-9ebe-4495-8aea-c3baf811f695-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.079381 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.079323    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.080336 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.080307    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25fq2\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-kube-api-access-25fq2\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.363500 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.363409    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:00.785997 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.785961    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"]
Oct 30 21:40:00.789986 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:40:00.789956    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f96548_9ebe_4495_8aea_c3baf811f695.slice/crio-a5d5bd3a27222886ca8ed0bd7acc0c873c9fcf16f7289ebf96688eef6ad4bd51 WatchSource:0}: Error finding container a5d5bd3a27222886ca8ed0bd7acc0c873c9fcf16f7289ebf96688eef6ad4bd51: Status 404 returned error can't find the container with id a5d5bd3a27222886ca8ed0bd7acc0c873c9fcf16f7289ebf96688eef6ad4bd51
Oct 30 21:40:00.792695 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.792657    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:00.792791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.792735    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:00.792791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:00.792770    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:01.048214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.048131    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"]
Oct 30 21:40:01.052667 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.052646    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.103776 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.103747    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-71aab5-openshift-default-dockercfg-hxtfl\""
Oct 30 21:40:01.154429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.154384    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"]
Oct 30 21:40:01.227283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.227248    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" event={"ID":"20f96548-9ebe-4495-8aea-c3baf811f695","Type":"ContainerStarted","Data":"e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e"}
Oct 30 21:40:01.227478 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.227309    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" event={"ID":"20f96548-9ebe-4495-8aea-c3baf811f695","Type":"ContainerStarted","Data":"a5d5bd3a27222886ca8ed0bd7acc0c873c9fcf16f7289ebf96688eef6ad4bd51"}
Oct 30 21:40:01.231172 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231152    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istiod-ca-cert\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231191    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-data\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231218    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-certs\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231278    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231334    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-token\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231441 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231422    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-podinfo\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231467    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q2t9\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-kube-api-access-8q2t9\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231561    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-envoy\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.231648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.231631    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-credential-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333332    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q2t9\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-kube-api-access-8q2t9\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333445    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-envoy\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333468    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-credential-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333592    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istiod-ca-cert\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333648    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-data\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333877 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333711    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-certs\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333877 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333747    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333877 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333769    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-token\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.333877 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333803    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-podinfo\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.334087 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.333991    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-credential-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.334251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.334222    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.334453 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.334429    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-certs\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.334548 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.334481    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istiod-ca-cert\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.334548 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.334496    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-data\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.336122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.336102    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-envoy\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.336257 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.336242    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-podinfo\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.352256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.352216    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podStartSLOduration=2.352204125 podStartE2EDuration="2.352204125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:01.348192437 +0000 UTC m=+1536.635914982" watchObservedRunningTime="2026-10-30 21:40:01.352204125 +0000 UTC m=+1536.639926694"
Oct 30 21:40:01.364208 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.364187    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:01.365801 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.365781    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:40:01.365885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.365819    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 21:40:01.413370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.398699    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-token\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.414407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.414119    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q2t9\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-kube-api-access-8q2t9\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:01.667481 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:01.667387    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:02.132898 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:02.132871    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"]
Oct 30 21:40:02.134933 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:40:02.134898    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8a0fb2_6236_41c4_9fde_34d97a9a71d3.slice/crio-5395c01e845181553d87b0ad614832e81ff8d94fc13dd1a172418b65c94bad0f WatchSource:0}: Error finding container 5395c01e845181553d87b0ad614832e81ff8d94fc13dd1a172418b65c94bad0f: Status 404 returned error can't find the container with id 5395c01e845181553d87b0ad614832e81ff8d94fc13dd1a172418b65c94bad0f
Oct 30 21:40:02.137717 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:02.137678    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:02.137804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:02.137770    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:02.137842 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:02.137817    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:02.236241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:02.236203    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" event={"ID":"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3","Type":"ContainerStarted","Data":"5395c01e845181553d87b0ad614832e81ff8d94fc13dd1a172418b65c94bad0f"}
Oct 30 21:40:02.363764 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:02.363731    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:40:02.363928 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:02.363780    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 21:40:03.245781 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:03.245741    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" event={"ID":"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3","Type":"ContainerStarted","Data":"c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17"}
Oct 30 21:40:03.364033 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:03.363698    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:40:03.364033 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:03.363776    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 21:40:03.442401 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:03.441992    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podStartSLOduration=2.441969608 podStartE2EDuration="2.441969608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:03.438642537 +0000 UTC m=+1538.726365097" watchObservedRunningTime="2026-10-30 21:40:03.441969608 +0000 UTC m=+1538.729692158"
Oct 30 21:40:03.669036 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:03.668964    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:03.669177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:03.669073    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:40:03.669177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:03.669106    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:40:04.363891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:04.363852    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:40:04.364290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:04.363920    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 21:40:04.668511 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:04.668418    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:40:04.668511 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:04.668480    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:40:05.364687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:05.364647    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:40:05.365082 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:05.364703    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 21:40:05.668029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:05.667941    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:40:05.668029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:05.667992    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:40:06.365255 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:06.365211    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:40:06.365771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:06.365286    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 21:40:06.668709 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:06.668626    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:40:06.668709 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:06.668685    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:40:07.366642 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:07.363749    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 21:40:07.366642 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:07.363811    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 21:40:07.669788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:07.668909    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:40:07.669788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:07.668999    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:40:08.369289 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:08.369253    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:08.369735 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:08.369686    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:08.370389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:08.370336    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 21:40:08.668217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:08.668127    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:40:08.668217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:08.668181    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:40:09.673240 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:09.673207    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:09.673656 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:09.673594    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:09.674222 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:09.674205    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 21:40:16.338711 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:40:16.338674    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:40:29.851823 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:29.851786    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"]
Oct 30 21:40:29.882003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:29.880854    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:29.934073 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:29.934037    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"]
Oct 30 21:40:30.026793 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.026762    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.026793 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.026798    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh277\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-kube-api-access-fh277\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.027046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.026928    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada0244e-09c2-44f7-9200-73b6641dc731-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.027046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.026956    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada0244e-09c2-44f7-9200-73b6641dc731-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.027046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.026984    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.027199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.027087    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.027199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.027111    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.027199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.027132    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.027374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.027244    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128091    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128170    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128189    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh277\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-kube-api-access-fh277\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128252    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada0244e-09c2-44f7-9200-73b6641dc731-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128269    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada0244e-09c2-44f7-9200-73b6641dc731-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128291    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128339    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128402    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128431    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128734    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.128945 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128925    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.129008 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.128962    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.129062 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.129033    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.129121 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.129066    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada0244e-09c2-44f7-9200-73b6641dc731-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.130435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.130413    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.130643 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.130626    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada0244e-09c2-44f7-9200-73b6641dc731-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.186253 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.186220    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.186422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.186380    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh277\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-kube-api-access-fh277\") pod \"maas-default-gateway-openshift-default-67bf986f99-n5tft\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.201251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.201228    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:30.620822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.620798    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"]
Oct 30 21:40:30.622821 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:40:30.622787    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada0244e_09c2_44f7_9200_73b6641dc731.slice/crio-8e4f7e30a1bc20ada5bc721a612154adc61894933248a6c4dac94d80a62739df WatchSource:0}: Error finding container 8e4f7e30a1bc20ada5bc721a612154adc61894933248a6c4dac94d80a62739df: Status 404 returned error can't find the container with id 8e4f7e30a1bc20ada5bc721a612154adc61894933248a6c4dac94d80a62739df
Oct 30 21:40:30.625262 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.625232    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:30.625325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.625301    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:30.625393 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:30.625331    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:31.463954 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:31.463913    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" event={"ID":"ada0244e-09c2-44f7-9200-73b6641dc731","Type":"ContainerStarted","Data":"4ae72963ce8b2cae13438657986fa37c05b78ab6684c1b6dbd9bcda0289a3882"}
Oct 30 21:40:31.464576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:31.464544    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" event={"ID":"ada0244e-09c2-44f7-9200-73b6641dc731","Type":"ContainerStarted","Data":"8e4f7e30a1bc20ada5bc721a612154adc61894933248a6c4dac94d80a62739df"}
Oct 30 21:40:31.627923 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:31.627855    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podStartSLOduration=2.627835807 podStartE2EDuration="2.627835807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:31.627104399 +0000 UTC m=+1566.914826948" watchObservedRunningTime="2026-10-30 21:40:31.627835807 +0000 UTC m=+1566.915558355"
Oct 30 21:40:32.211368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:32.202745    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:32.214510 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:32.214479    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-n5tft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:40:32.214643 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:32.214537    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:40:33.210370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:33.209683    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-n5tft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:40:33.210370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:33.209743    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:40:34.209370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:34.208771    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-n5tft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:40:34.209370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:34.208829    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:40:35.209371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:35.203732    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-n5tft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:40:35.209371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:35.203792    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:40:36.212375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:36.206760    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-n5tft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:40:36.212375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:36.206850    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:40:37.201846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:37.201811    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-n5tft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:40:37.202023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:37.201861    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:40:38.201952 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:38.201917    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-n5tft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:40:38.202377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:38.201980    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:40:39.255855 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:39.255819    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:39.256532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:39.256508    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:39.257396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:39.257374    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:40:44.947732 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:44.947697    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"]
Oct 30 21:40:44.952013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:44.951986    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052431    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052649 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052489    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052649 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052621    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052775 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052676    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052775 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052720    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052775 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052749    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk4ch\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-kube-api-access-wk4ch\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052936 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052867    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052943    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.052991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.052967    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.053623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.053598    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"]
Oct 30 21:40:45.154123 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154095    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154156    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154172    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154210    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154229    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154259    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154299    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154690 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154340    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154690 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154400    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk4ch\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-kube-api-access-wk4ch\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154690 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154632    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154861 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154777    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154861 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154846    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.154973 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.154943    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.155193 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.155166    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.156472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.156443    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.156670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.156655    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.212426 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.212401    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.212595 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.212577    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk4ch\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-kube-api-access-wk4ch\") pod \"maas-default-gateway-openshift-default-67bf986f99-j5r89\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.268994 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.268970    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:45.696143 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.696113    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"]
Oct 30 21:40:45.698147 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:40:45.698119    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b16019b_1086_4ae4_baf6_2cd27be3bcfe.slice/crio-1ccf73817222f8027e3592607cb99e16c5a356e2fd2fb17454fcdc46619c402e WatchSource:0}: Error finding container 1ccf73817222f8027e3592607cb99e16c5a356e2fd2fb17454fcdc46619c402e: Status 404 returned error can't find the container with id 1ccf73817222f8027e3592607cb99e16c5a356e2fd2fb17454fcdc46619c402e
Oct 30 21:40:45.700492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.700457    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:45.700558 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.700527    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:45.700558 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:45.700554    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:40:46.341520 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:40:46.341478    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:40:46.622914 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:46.622820    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" event={"ID":"8b16019b-1086-4ae4-baf6-2cd27be3bcfe","Type":"ContainerStarted","Data":"7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f"}
Oct 30 21:40:46.622914 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:46.622883    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" event={"ID":"8b16019b-1086-4ae4-baf6-2cd27be3bcfe","Type":"ContainerStarted","Data":"1ccf73817222f8027e3592607cb99e16c5a356e2fd2fb17454fcdc46619c402e"}
Oct 30 21:40:46.832119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:46.832063    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podStartSLOduration=2.832049863 podStartE2EDuration="2.832049863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:40:46.783915844 +0000 UTC m=+1582.071638392" watchObservedRunningTime="2026-10-30 21:40:46.832049863 +0000 UTC m=+1582.119772472"
Oct 30 21:40:47.270110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:47.270076    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:47.271790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:47.271768    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-j5r89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:40:47.271867 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:47.271817    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:40:48.269520 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:48.269484    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-j5r89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:40:48.269988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:48.269543    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:40:49.269706 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:49.269667    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-j5r89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:40:49.270080 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:49.269725    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:40:50.269691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:50.269658    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-j5r89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:40:50.269881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:50.269711    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:40:51.270187 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:51.270156    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-j5r89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:40:51.270589 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:51.270219    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:40:52.269819 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:52.269775    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-j5r89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:40:52.270010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:52.269845    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:40:53.271804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:53.270706    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-j5r89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:40:53.271804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:53.270780    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:40:54.273857 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:54.273828    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:54.274304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:54.274282    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:40:54.274964 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:40:54.274942    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:41:00.861158 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.861125    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"]
Oct 30 21:41:00.871598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.871577    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.923592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.923567    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"]
Oct 30 21:41:00.961048 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961014    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-socket\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961187 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961126    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-certs\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961187 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961176    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-podinfo\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961284 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961204    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f17ad69-4769-4a86-84da-5c3f9f2df240-istiod-ca-cert\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961321 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961303    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-token\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961384 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961369    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-credential-socket\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961546 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961524    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-data\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961749 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961724    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-envoy\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.961800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.961785    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq2f6\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-kube-api-access-jq2f6\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:00.967287 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:00.967267    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-19dd4f-openshift-default-dockercfg-rbrph\""
Oct 30 21:41:01.062826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.062796    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-envoy\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063007 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.062847    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq2f6\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-kube-api-access-jq2f6\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063007 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.062930    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-socket\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063007 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.062980    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-certs\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063034    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-podinfo\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063076    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f17ad69-4769-4a86-84da-5c3f9f2df240-istiod-ca-cert\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063127    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-token\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063186    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-credential-socket\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063275    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-data\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063386    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-socket\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063478    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-certs\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063673 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063654    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-data\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063760 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063738    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-credential-socket\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.063916 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.063892    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f17ad69-4769-4a86-84da-5c3f9f2df240-istiod-ca-cert\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.065226 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.065185    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-envoy\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.065481 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.065463    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-podinfo\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.120652 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.120593    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq2f6\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-kube-api-access-jq2f6\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.133817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.133787    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-token\") pod \"e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") " pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.186570 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.186543    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:01.611803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.611764    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"]
Oct 30 21:41:01.614590 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:41:01.614553    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f17ad69_4769_4a86_84da_5c3f9f2df240.slice/crio-76546e8936df4854307eca0cb4771081c2b4646ff7593528770dd9ca277891e1 WatchSource:0}: Error finding container 76546e8936df4854307eca0cb4771081c2b4646ff7593528770dd9ca277891e1: Status 404 returned error can't find the container with id 76546e8936df4854307eca0cb4771081c2b4646ff7593528770dd9ca277891e1
Oct 30 21:41:01.617196 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.617157    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:01.617272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.617249    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:01.617315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.617293    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:41:01.711686 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.711653    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" event={"ID":"4f17ad69-4769-4a86-84da-5c3f9f2df240","Type":"ContainerStarted","Data":"e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d"}
Oct 30 21:41:01.711874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:01.711714    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" event={"ID":"4f17ad69-4769-4a86-84da-5c3f9f2df240","Type":"ContainerStarted","Data":"76546e8936df4854307eca0cb4771081c2b4646ff7593528770dd9ca277891e1"}
Oct 30 21:41:02.187167 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:02.187023    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:02.200799 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:02.200764    5817 patch_prober.go:29] interesting pod/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:41:02.201578 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:02.201536    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:41:03.187447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:03.187411    5817 patch_prober.go:29] interesting pod/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:41:03.187833 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:03.187465    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:41:04.187022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:04.186987    5817 patch_prober.go:29] interesting pod/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:41:04.187199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:04.187036    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:41:05.187332 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:05.187299    5817 patch_prober.go:29] interesting pod/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:41:05.187748 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:05.187371    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:41:06.187615 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:06.187581    5817 patch_prober.go:29] interesting pod/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:41:06.188004 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:06.187644    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:41:07.187924 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:07.187888    5817 patch_prober.go:29] interesting pod/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:41:07.188320 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:07.187944    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:41:08.187814 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:08.187782    5817 patch_prober.go:29] interesting pod/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:41:08.187982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:08.187836    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:41:09.191578 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:09.191545    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:09.192050 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:09.191824    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:09.192658 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:09.192640    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:41:09.353793 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:09.353745    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podStartSLOduration=9.353731827 podStartE2EDuration="9.353731827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:01.834658455 +0000 UTC m=+1597.122381002" watchObservedRunningTime="2026-10-30 21:41:09.353731827 +0000 UTC m=+1604.641454421"
Oct 30 21:41:16.338233 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:41:16.338192    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:41:29.889789 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:29.889756    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"]
Oct 30 21:41:29.894388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:29.894335    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:29.950918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:29.950876    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"]
Oct 30 21:41:29.993765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:29.993738    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-991335\"/\"distinct-w5-991335-kserve-self-signed-certs\""
Oct 30 21:41:30.002129 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.002106    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-home\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.002227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.002136    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-tmp-dir\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.002303 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.002283    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289aead-2be5-4573-9f7b-52be0b91b408-tls-certs\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.002451 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.002434    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-579bt\" (UniqueName: \"kubernetes.io/projected/c289aead-2be5-4573-9f7b-52be0b91b408-kube-api-access-579bt\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.002523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.002477    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-dshm\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.002567 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.002519    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-model-cache\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104032    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289aead-2be5-4573-9f7b-52be0b91b408-tls-certs\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104118    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-579bt\" (UniqueName: \"kubernetes.io/projected/c289aead-2be5-4573-9f7b-52be0b91b408-kube-api-access-579bt\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104148    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-dshm\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104173    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-model-cache\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104247    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-home\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104263    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-tmp-dir\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104629    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-model-cache\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104737 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104675    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-tmp-dir\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.104737 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.104694    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-home\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.106279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.106258    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-dshm\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.106543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.106525    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289aead-2be5-4573-9f7b-52be0b91b408-tls-certs\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.168099 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.168033    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-579bt\" (UniqueName: \"kubernetes.io/projected/c289aead-2be5-4573-9f7b-52be0b91b408-kube-api-access-579bt\") pod \"distinct-w5-991335-kserve-68ddbd7fcd-wdk2d\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") " pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.210459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.210430    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:30.620905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.620874    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"]
Oct 30 21:41:30.622164 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:41:30.622139    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc289aead_2be5_4573_9f7b_52be0b91b408.slice/crio-067dee3c8dc59ebe2619292f0531e37929ec63aae6785e792b5549df6e1501a8 WatchSource:0}: Error finding container 067dee3c8dc59ebe2619292f0531e37929ec63aae6785e792b5549df6e1501a8: Status 404 returned error can't find the container with id 067dee3c8dc59ebe2619292f0531e37929ec63aae6785e792b5549df6e1501a8
Oct 30 21:41:30.917429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.917306    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" event={"ID":"c289aead-2be5-4573-9f7b-52be0b91b408","Type":"ContainerStarted","Data":"462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469"}
Oct 30 21:41:30.917429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:30.917376    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" event={"ID":"c289aead-2be5-4573-9f7b-52be0b91b408","Type":"ContainerStarted","Data":"067dee3c8dc59ebe2619292f0531e37929ec63aae6785e792b5549df6e1501a8"}
Oct 30 21:41:31.030171 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:31.030120    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podStartSLOduration=2.030107097 podStartE2EDuration="2.030107097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:31.028694765 +0000 UTC m=+1626.316417315" watchObservedRunningTime="2026-10-30 21:41:31.030107097 +0000 UTC m=+1626.317829681"
Oct 30 21:41:31.210767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:31.210738    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:40.211509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:40.211471    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:40.223645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:40.223613    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:40.986484 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:40.986451    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:41:44.607910 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.607872    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"]
Oct 30 21:41:44.611904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.611879    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.671050 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.671019    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-554f46\"/\"distinct-w4-554f46-kserve-self-signed-certs\""
Oct 30 21:41:44.671240 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.671181    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-554f46\"/\"default-dockercfg-dlzxb\""
Oct 30 21:41:44.679764 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.679738    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"]
Oct 30 21:41:44.717889 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.717860    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-554f46\"/\"openshift-service-ca.crt\""
Oct 30 21:41:44.718024 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.717944    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-554f46\"/\"kube-root-ca.crt\""
Oct 30 21:41:44.781018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.780978    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdk4k\" (UniqueName: \"kubernetes.io/projected/865b13f8-87cb-496f-b90f-080ed2aaff6e-kube-api-access-hdk4k\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.781217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.781039    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-model-cache\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.781290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.781230    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-home\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.781422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.781402    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b13f8-87cb-496f-b90f-080ed2aaff6e-tls-certs\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.781504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.781463    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-dshm\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.781564 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.781521    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-tmp-dir\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883140    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b13f8-87cb-496f-b90f-080ed2aaff6e-tls-certs\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883185    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-dshm\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883226    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-tmp-dir\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883568 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883268    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdk4k\" (UniqueName: \"kubernetes.io/projected/865b13f8-87cb-496f-b90f-080ed2aaff6e-kube-api-access-hdk4k\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883568 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883289    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-model-cache\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883568 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883383    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-home\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883757 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883723    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-tmp-dir\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883766    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-model-cache\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.883838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.883804    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-home\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.885537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.885515    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-dshm\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.885708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.885691    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b13f8-87cb-496f-b90f-080ed2aaff6e-tls-certs\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:44.942131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:44.942097    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdk4k\" (UniqueName: \"kubernetes.io/projected/865b13f8-87cb-496f-b90f-080ed2aaff6e-kube-api-access-hdk4k\") pod \"distinct-w4-554f46-kserve-795bf7c4c9-jgz7j\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") " pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:45.224879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:45.224845    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:45.598279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:45.598247    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"]
Oct 30 21:41:45.599906 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:41:45.599868    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod865b13f8_87cb_496f_b90f_080ed2aaff6e.slice/crio-a896a4e05fdf3e25eb9342506aa80526f1cfd086c72da416246b7c6376b771cd WatchSource:0}: Error finding container a896a4e05fdf3e25eb9342506aa80526f1cfd086c72da416246b7c6376b771cd: Status 404 returned error can't find the container with id a896a4e05fdf3e25eb9342506aa80526f1cfd086c72da416246b7c6376b771cd
Oct 30 21:41:46.006230 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:46.006198    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" event={"ID":"865b13f8-87cb-496f-b90f-080ed2aaff6e","Type":"ContainerStarted","Data":"de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8"}
Oct 30 21:41:46.006657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:46.006237    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" event={"ID":"865b13f8-87cb-496f-b90f-080ed2aaff6e","Type":"ContainerStarted","Data":"a896a4e05fdf3e25eb9342506aa80526f1cfd086c72da416246b7c6376b771cd"}
Oct 30 21:41:46.161705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:46.161645    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podStartSLOduration=2.161626221 podStartE2EDuration="2.161626221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:46.159636782 +0000 UTC m=+1641.447359339" watchObservedRunningTime="2026-10-30 21:41:46.161626221 +0000 UTC m=+1641.449348767"
Oct 30 21:41:46.225402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:46.225359    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:46.354471 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:41:46.354378    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:41:55.225867 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:55.225835    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:55.238077 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:55.238053    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:56.088479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:56.088441    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:41:58.942757 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:58.942719    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"]
Oct 30 21:41:58.947163 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:58.947142    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.046584 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.046555    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-a2dced\"/\"kube-root-ca.crt\""
Oct 30 21:41:59.046808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.046776    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-a2dced\"/\"openshift-service-ca.crt\""
Oct 30 21:41:59.046922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.046825    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-a2dced\"/\"default-dockercfg-smfxk\""
Oct 30 21:41:59.046922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.046857    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-a2dced\"/\"distinct-w6-a2dced-kserve-self-signed-certs\""
Oct 30 21:41:59.050325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.050307    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"]
Oct 30 21:41:59.069598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.069572    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tls-certs\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.069726 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.069638    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-home\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.069815 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.069796    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsjft\" (UniqueName: \"kubernetes.io/projected/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-kube-api-access-bsjft\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.069879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.069863    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-model-cache\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.069931 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.069913    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-dshm\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.070059 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.070043    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tmp-dir\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.170918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.170887    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tmp-dir\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.170918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.170919    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tls-certs\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.171161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.170963    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-home\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.171161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.171021    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsjft\" (UniqueName: \"kubernetes.io/projected/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-kube-api-access-bsjft\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.171161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.171068    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-model-cache\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.171161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.171096    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-dshm\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.171407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.171307    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tmp-dir\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.171407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.171376    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-home\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.171523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.171443    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-model-cache\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.173369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.173316    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-dshm\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.173472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.173438    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tls-certs\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.232154 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.232118    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsjft\" (UniqueName: \"kubernetes.io/projected/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-kube-api-access-bsjft\") pod \"distinct-w6-a2dced-kserve-7dd455cdd-bwjkt\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") " pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.260564 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.260525    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:41:59.636432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:41:59.636402    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"]
Oct 30 21:41:59.636649 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:41:59.636632    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod313a4a3a_5ea1_4090_b8d6_40f181ffd3aa.slice/crio-407e9a209574d9abedfe87878f9388108be572d54d32b5239bfa9d08874224d1 WatchSource:0}: Error finding container 407e9a209574d9abedfe87878f9388108be572d54d32b5239bfa9d08874224d1: Status 404 returned error can't find the container with id 407e9a209574d9abedfe87878f9388108be572d54d32b5239bfa9d08874224d1
Oct 30 21:42:00.118333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:00.118301    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" event={"ID":"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa","Type":"ContainerStarted","Data":"eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857"}
Oct 30 21:42:00.118333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:00.118355    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" event={"ID":"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa","Type":"ContainerStarted","Data":"407e9a209574d9abedfe87878f9388108be572d54d32b5239bfa9d08874224d1"}
Oct 30 21:42:00.243415 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:00.242513    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podStartSLOduration=2.242492824 podStartE2EDuration="2.242492824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:00.241803788 +0000 UTC m=+1655.529526335" watchObservedRunningTime="2026-10-30 21:42:00.242492824 +0000 UTC m=+1655.530215372"
Oct 30 21:42:00.264513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:00.264440    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:42:08.356717 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.356680    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"]
Oct 30 21:42:08.360870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.360849    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.457225 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.457191    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-71aab5\"/\"distinct-w3-71aab5-kserve-self-signed-certs\""
Oct 30 21:42:08.459956 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.459932    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"]
Oct 30 21:42:08.509107 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.509060    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-266wt\" (UniqueName: \"kubernetes.io/projected/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-kube-api-access-266wt\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.509314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.509146    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-dshm\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.509404 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.509322    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tls-certs\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.509404 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.509389    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-model-cache\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.509489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.509463    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-home\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.509616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.509548    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tmp-dir\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.612741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.612561    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tls-certs\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.612741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.612644    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-model-cache\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.612741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.612709    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-home\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.613002 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.612800    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tmp-dir\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.613002 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.612898    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-266wt\" (UniqueName: \"kubernetes.io/projected/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-kube-api-access-266wt\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.613002 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.612950    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-dshm\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.616095 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.616068    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-model-cache\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.616592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.616097    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tmp-dir\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.616592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.616340    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-home\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.620484 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.620456    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-dshm\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.624303 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.624277    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tls-certs\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.675746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.675711    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-266wt\" (UniqueName: \"kubernetes.io/projected/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-kube-api-access-266wt\") pod \"distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") " pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:08.974325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:08.974299    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:09.261442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:09.261360    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:42:09.273520 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:09.273494    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:42:09.348207 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:42:09.348166    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9c6d767_7ebe_46d5_8de8_fc46d3eaa530.slice/crio-c54de483160b56b67c5b750e4c19d7995f747447fbcf498141287e17539f7979 WatchSource:0}: Error finding container c54de483160b56b67c5b750e4c19d7995f747447fbcf498141287e17539f7979: Status 404 returned error can't find the container with id c54de483160b56b67c5b750e4c19d7995f747447fbcf498141287e17539f7979
Oct 30 21:42:09.373151 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:09.373125    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"]
Oct 30 21:42:10.180497 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:10.180460    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" event={"ID":"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530","Type":"ContainerStarted","Data":"800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928"}
Oct 30 21:42:10.180699 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:10.180508    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" event={"ID":"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530","Type":"ContainerStarted","Data":"c54de483160b56b67c5b750e4c19d7995f747447fbcf498141287e17539f7979"}
Oct 30 21:42:10.191613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:10.191590    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:42:10.294269 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:10.294223    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podStartSLOduration=2.294209529 podStartE2EDuration="2.294209529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:10.292998115 +0000 UTC m=+1665.580720660" watchObservedRunningTime="2026-10-30 21:42:10.294209529 +0000 UTC m=+1665.581932075"
Oct 30 21:42:10.974594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:10.974558    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:16.339131 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:42:16.339092    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:42:17.502474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.502433    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"]
Oct 30 21:42:17.506657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.506636    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.617370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.616544    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"]
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641179    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641278    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641335    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641424    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544e7dca-6988-491c-907b-e903dafe5c65-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641464    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544e7dca-6988-491c-907b-e903dafe5c65-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641501    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641537    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641659    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28fsf\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-kube-api-access-28fsf\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.643449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.641712    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743287    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743531 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743357    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743531 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743404    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544e7dca-6988-491c-907b-e903dafe5c65-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743531 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743430    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544e7dca-6988-491c-907b-e903dafe5c65-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743531 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743465    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743531 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743506    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743617    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28fsf\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-kube-api-access-28fsf\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743671    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743743    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.743940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.743881    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.744098 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.744073    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.744167 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.744129    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.744291 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.744261    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544e7dca-6988-491c-907b-e903dafe5c65-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.744387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.744317    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.746316 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.746288    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544e7dca-6988-491c-907b-e903dafe5c65-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.746465 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.746368    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.802480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.802407    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.802638 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.802613    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28fsf\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-kube-api-access-28fsf\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:17.822813 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:17.822774    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:18.265724 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:42:18.265700    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod544e7dca_6988_491c_907b_e903dafe5c65.slice/crio-04cc2165cc98aae2beaed988773bb6a7e5a28ffb0bcb835ac19b1a0badc6cdea WatchSource:0}: Error finding container 04cc2165cc98aae2beaed988773bb6a7e5a28ffb0bcb835ac19b1a0badc6cdea: Status 404 returned error can't find the container with id 04cc2165cc98aae2beaed988773bb6a7e5a28ffb0bcb835ac19b1a0badc6cdea
Oct 30 21:42:18.265935 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:18.265913    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"]
Oct 30 21:42:18.268448 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:18.268412    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:18.268560 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:18.268478    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:18.268560 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:18.268506    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:18.974546 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:18.974516    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:18.987614 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:18.987593    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:19.252476 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:19.252387    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" event={"ID":"544e7dca-6988-491c-907b-e903dafe5c65","Type":"ContainerStarted","Data":"a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51"}
Oct 30 21:42:19.252476 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:19.252454    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" event={"ID":"544e7dca-6988-491c-907b-e903dafe5c65","Type":"ContainerStarted","Data":"04cc2165cc98aae2beaed988773bb6a7e5a28ffb0bcb835ac19b1a0badc6cdea"}
Oct 30 21:42:19.264045 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:19.264016    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:42:19.382787 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:19.382732    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podStartSLOduration=2.382718654 podStartE2EDuration="2.382718654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:19.38019717 +0000 UTC m=+1674.667919717" watchObservedRunningTime="2026-10-30 21:42:19.382718654 +0000 UTC m=+1674.670441199"
Oct 30 21:42:19.823230 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:19.823189    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:19.824424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:19.824400    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:42:19.824536 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:19.824444    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:42:20.824111 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:20.824076    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:42:20.824492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:20.824146    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:42:21.824034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:21.823995    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:42:21.824214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:21.824050    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:42:22.823890 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:22.823850    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:42:22.824073 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:22.823904    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:42:23.823515 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:23.823471    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:42:23.823890 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:23.823535    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:42:24.823775 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:24.823731    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:42:24.824274 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:24.823807    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:42:25.827598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:25.827567    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:25.827976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:25.827846    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:25.828583 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:25.828563    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:42:30.455979 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.455939    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"]
Oct 30 21:42:30.461245 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.461225    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.543799 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.543767    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.543932 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.543817    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.543984 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.543933    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.544047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.544030    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.544109 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.544093    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.544155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.544116    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.544324 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.544303    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.544423 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.544403    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.544481 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.544471    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9kk2\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-kube-api-access-n9kk2\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.562552 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.562092    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"]
Oct 30 21:42:30.645544 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645515    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645560    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645590    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9kk2\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-kube-api-access-n9kk2\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645612    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645641    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645899 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645697    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645899 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645787    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645899 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645851    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.645899 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.645890    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.646119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.646032    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.646181 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.646148    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.646243 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.646213    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.646333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.646311    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.646462 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.646445    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.648106 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.648084    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.648361 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.648325    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.703945 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.703909    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.704116 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.703973    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9kk2\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-kube-api-access-n9kk2\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:30.774331 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:30.774225    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:31.197651 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.197606    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"]
Oct 30 21:42:31.199129 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:42:31.199100    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db80d85_129d_4d02_b23b_1be8eaa5d1fe.slice/crio-1a3650557de808cab73de199c43be1ed38182b6138084b0a4603c13cb8083443 WatchSource:0}: Error finding container 1a3650557de808cab73de199c43be1ed38182b6138084b0a4603c13cb8083443: Status 404 returned error can't find the container with id 1a3650557de808cab73de199c43be1ed38182b6138084b0a4603c13cb8083443
Oct 30 21:42:31.201469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.201439    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:31.201537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.201503    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:31.201537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.201533    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:31.327642 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.327604    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" event={"ID":"7db80d85-129d-4d02-b23b-1be8eaa5d1fe","Type":"ContainerStarted","Data":"973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b"}
Oct 30 21:42:31.327848 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.327660    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" event={"ID":"7db80d85-129d-4d02-b23b-1be8eaa5d1fe","Type":"ContainerStarted","Data":"1a3650557de808cab73de199c43be1ed38182b6138084b0a4603c13cb8083443"}
Oct 30 21:42:31.447908 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.447798    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podStartSLOduration=1.447779576 podStartE2EDuration="1.447779576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:31.442737439 +0000 UTC m=+1686.730459985" watchObservedRunningTime="2026-10-30 21:42:31.447779576 +0000 UTC m=+1686.735502120"
Oct 30 21:42:31.775094 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:31.775059    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:32.776535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:32.776503    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:42:32.776960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:32.776555    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:42:33.774800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:33.774767    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:42:33.774995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:33.774820    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:42:34.775533 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:34.775492    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:42:34.775891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:34.775562    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:42:35.774590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:35.774554    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:42:35.774789 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:35.774613    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:42:36.774854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:36.774819    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:42:36.775384 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:36.774875    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:42:37.775500 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:37.775462    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:42:37.775876 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:37.775517    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:42:38.780906 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:38.780875    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:38.781299 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:38.781266    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:38.782110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:38.782089    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:42:46.338074 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:42:46.338036    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:42:46.421889 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.421857    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"]
Oct 30 21:42:46.426076 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.426056    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.474682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.474651    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.474825 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.474690    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.474879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.474829    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.474964 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.474950    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxzhl\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-kube-api-access-qxzhl\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.475010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.474977    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.475062 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.475046    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.475157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.475137    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.475234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.475218    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1160449-671a-4ba2-ae4d-aaa6a3692714-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.475322 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.475310    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.528573 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.528541    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"]
Oct 30 21:42:46.576822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.576785    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.576977 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.576894    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzhl\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-kube-api-access-qxzhl\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.576977 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.576915    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.576977 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.576955    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.576983    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577020    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1160449-671a-4ba2-ae4d-aaa6a3692714-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577050    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577087    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577115    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577380 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577207    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577380 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577232    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577486    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577534    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.577873 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.577853    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1160449-671a-4ba2-ae4d-aaa6a3692714-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.579302 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.579280    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.579561 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.579544    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.640047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.639323    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxzhl\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-kube-api-access-qxzhl\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.648207 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.648182    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:46.743027 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:46.742994    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:47.174493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.174470    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"]
Oct 30 21:42:47.175703 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:42:47.175673    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1160449_671a_4ba2_ae4d_aaa6a3692714.slice/crio-8df58d8e70c5b1fa5667cb462d532da9f191a7eba67785844e36bb9997d57684 WatchSource:0}: Error finding container 8df58d8e70c5b1fa5667cb462d532da9f191a7eba67785844e36bb9997d57684: Status 404 returned error can't find the container with id 8df58d8e70c5b1fa5667cb462d532da9f191a7eba67785844e36bb9997d57684
Oct 30 21:42:47.178221 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.178186    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:47.178290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.178257    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:47.178330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.178292    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:42:47.431526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.431416    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" event={"ID":"b1160449-671a-4ba2-ae4d-aaa6a3692714","Type":"ContainerStarted","Data":"e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2"}
Oct 30 21:42:47.431526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.431484    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" event={"ID":"b1160449-671a-4ba2-ae4d-aaa6a3692714","Type":"ContainerStarted","Data":"8df58d8e70c5b1fa5667cb462d532da9f191a7eba67785844e36bb9997d57684"}
Oct 30 21:42:47.551932 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.551877    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podStartSLOduration=1.5518630359999999 podStartE2EDuration="1.551863036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:47.549055316 +0000 UTC m=+1702.836777864" watchObservedRunningTime="2026-10-30 21:42:47.551863036 +0000 UTC m=+1702.839585581"
Oct 30 21:42:47.743232 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:47.743198    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:48.746008 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:48.745974    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:42:48.746425 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:48.746026    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:42:49.743634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:49.743600    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:42:49.743822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:49.743657    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:42:50.744258 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:50.744226    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:42:50.744678 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:50.744283    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:42:51.743340 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:51.743304    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:42:51.743558 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:51.743386    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:42:52.743274 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:52.743238    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:42:52.743704 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:52.743288    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:42:53.743517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:53.743473    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:42:53.743907 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:53.743562    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:42:54.747543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:54.747515    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:54.747978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:54.747828    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:42:54.748693 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:42:54.748673    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:43:16.339090 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:43:16.339048    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:43:17.339155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.339113    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"]
Oct 30 21:43:17.343912 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.343885    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.445557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.445523    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"]
Oct 30 21:43:17.518687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.518641    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.518879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.518694    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.518879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.518825    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcc2z\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-kube-api-access-dcc2z\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.518879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.518859    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.519020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.519006    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f507f65-d132-4b31-acfa-0cc08bd2a732-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.519058 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.519045    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.519109 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.519078    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.519190 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.519176    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.519233 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.519205    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.620896 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.620804    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcc2z\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-kube-api-access-dcc2z\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.620896 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.620843    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.620936    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f507f65-d132-4b31-acfa-0cc08bd2a732-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.620977    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621008    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621041    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621073    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621127    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621486 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621159    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621575 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621538    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621568    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.621929 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.621903    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f507f65-d132-4b31-acfa-0cc08bd2a732-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.622065 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.622041    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.622138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.622088    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.623785 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.623753    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.624241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.624222    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.679296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.679268    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.679524 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.679495    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcc2z\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-kube-api-access-dcc2z\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:17.958758 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:17.958722    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:18.381626 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.381554    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"]
Oct 30 21:43:18.383039 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:43:18.383014    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f507f65_d132_4b31_acfa_0cc08bd2a732.slice/crio-dafc4850af2bf169dffee1063dce1ef971b5be7ae16bfb607b7211ef034d7de2 WatchSource:0}: Error finding container dafc4850af2bf169dffee1063dce1ef971b5be7ae16bfb607b7211ef034d7de2: Status 404 returned error can't find the container with id dafc4850af2bf169dffee1063dce1ef971b5be7ae16bfb607b7211ef034d7de2
Oct 30 21:43:18.385469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.385434    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:18.385563 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.385504    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:18.385563 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.385536    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:43:18.631259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.631180    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" event={"ID":"1f507f65-d132-4b31-acfa-0cc08bd2a732","Type":"ContainerStarted","Data":"465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508"}
Oct 30 21:43:18.631259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.631254    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" event={"ID":"1f507f65-d132-4b31-acfa-0cc08bd2a732","Type":"ContainerStarted","Data":"dafc4850af2bf169dffee1063dce1ef971b5be7ae16bfb607b7211ef034d7de2"}
Oct 30 21:43:18.749003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.748955    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podStartSLOduration=1.748940605 podStartE2EDuration="1.748940605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:18.745640873 +0000 UTC m=+1734.033363437" watchObservedRunningTime="2026-10-30 21:43:18.748940605 +0000 UTC m=+1734.036663150"
Oct 30 21:43:18.959291 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:18.959261    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:19.960879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:19.960847    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:43:19.961242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:19.960894    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:43:20.959037 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:20.959000    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:43:20.959213 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:20.959070    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:43:21.959880 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:21.959846    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:43:21.960277 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:21.959915    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:43:22.959777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:22.959740    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:43:22.959946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:22.959802    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:43:23.959185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:23.959148    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:43:23.959377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:23.959208    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:43:24.959529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:24.959480    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:43:24.959951 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:24.959568    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:43:25.964489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:25.964446    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:25.964981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:25.964804    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:25.965593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:43:25.965574    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:43:46.338083 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:43:46.338043    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:44:16.338012 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:44:16.337977    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:44:26.208216 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.208183    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"]
Oct 30 21:44:26.212660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.212639    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.286680 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.286649    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ed76b8\"/\"default-dockercfg-wpc6v\""
Oct 30 21:44:26.313428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.313385    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-ed76b8\"/\"distinct-2-w2-ed76b8-kserve-self-signed-certs\""
Oct 30 21:44:26.313428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.313389    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ed76b8\"/\"openshift-service-ca.crt\""
Oct 30 21:44:26.313428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.313399    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-ed76b8\"/\"kube-root-ca.crt\""
Oct 30 21:44:26.316526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.316504    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"]
Oct 30 21:44:26.383504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.383471    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-dshm\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.383632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.383516    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qqg4\" (UniqueName: \"kubernetes.io/projected/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-kube-api-access-4qqg4\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.383632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.383553    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-home\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.383722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.383643    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tmp-dir\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.383722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.383679    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tls-certs\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.383722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.383704    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-model-cache\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485052 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.484967    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-dshm\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485052 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485009    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqg4\" (UniqueName: \"kubernetes.io/projected/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-kube-api-access-4qqg4\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485052 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485034    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-home\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485364 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485063    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tmp-dir\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485364 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485086    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tls-certs\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485364 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485109    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-model-cache\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485555    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-home\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485640 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485567    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tmp-dir\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.485640 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.485624    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-model-cache\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.487518 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.487492    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-dshm\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.487634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.487616    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tls-certs\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.543393 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.543364    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qqg4\" (UniqueName: \"kubernetes.io/projected/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-kube-api-access-4qqg4\") pod \"distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") " pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:26.828136 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:26.828035    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:27.212356 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:27.212320    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"]
Oct 30 21:44:27.214241 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:44:27.214207    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b5efc58_2ff8_4a72_a352_925e7bbe3a24.slice/crio-57df3b38b0591e2ceaa59907beec03eafb51f327b8f5249b5742891dc04b19f3 WatchSource:0}: Error finding container 57df3b38b0591e2ceaa59907beec03eafb51f327b8f5249b5742891dc04b19f3: Status 404 returned error can't find the container with id 57df3b38b0591e2ceaa59907beec03eafb51f327b8f5249b5742891dc04b19f3
Oct 30 21:44:28.123216 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:28.123183    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" event={"ID":"8b5efc58-2ff8-4a72-a352-925e7bbe3a24","Type":"ContainerStarted","Data":"85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65"}
Oct 30 21:44:28.123403 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:28.123224    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" event={"ID":"8b5efc58-2ff8-4a72-a352-925e7bbe3a24","Type":"ContainerStarted","Data":"57df3b38b0591e2ceaa59907beec03eafb51f327b8f5249b5742891dc04b19f3"}
Oct 30 21:44:28.242027 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:28.241978    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podStartSLOduration=2.241963103 podStartE2EDuration="2.241963103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:28.238851466 +0000 UTC m=+1803.526574005" watchObservedRunningTime="2026-10-30 21:44:28.241963103 +0000 UTC m=+1803.529685650"
Oct 30 21:44:28.828594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:28.828566    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:32.932448 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.932377    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"]
Oct 30 21:44:32.936705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.936685    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983431    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d84241f-3219-4f7c-9af6-084c34f8b986-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983493    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983630    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983673    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983703    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk8j2\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-kube-api-access-jk8j2\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983790    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983837 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983826    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.983956 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.983939    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:32.984022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:32.984009    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085396    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085618 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085507    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085618 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085585    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085750 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085730    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d84241f-3219-4f7c-9af6-084c34f8b986-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085806 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085769    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085862 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085836    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085865    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.085918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085893    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk8j2\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-kube-api-access-jk8j2\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.086023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085965    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.086023 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.085992    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.086279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.086256    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.086666 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.086640    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.087074 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.087048    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d84241f-3219-4f7c-9af6-084c34f8b986-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.087538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.087512    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.091420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.088707    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.091420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.088914    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.145254 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.145206    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.145625 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.145598    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk8j2\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-kube-api-access-jk8j2\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.150305 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.150274    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"]
Oct 30 21:44:33.251323 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.251284    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:33.536590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.536471    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"]
Oct 30 21:44:33.542242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.542216    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.592668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.592638    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.592826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.592674    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.592826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.592701    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.592913 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.592850    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.592913 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.592889    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.592913 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.592906    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r48gf\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-kube-api-access-r48gf\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.593067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.593046    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.593199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.593185    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.593267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.593255    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.635804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.635763    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"]
Oct 30 21:44:33.694703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.694668    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.694858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.694761    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.694858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.694818    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.694858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.694844    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.694882    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.694932    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.694987    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.695014    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r48gf\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-kube-api-access-r48gf\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.695071    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-certs\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.695089    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.695138    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.695202    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-credential-socket\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695600 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.695450    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-data\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.695734 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.695715    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istiod-ca-cert\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.697227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.697206    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-envoy\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.697644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.697544    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-podinfo\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.704925 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.704905    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"]
Oct 30 21:44:33.728643 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:44:33.728613    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d84241f_3219_4f7c_9af6_084c34f8b986.slice/crio-0316b56d9e8177cee54ff613423e20180e3c853c6e7db57899ac6b4e327ccfd4 WatchSource:0}: Error finding container 0316b56d9e8177cee54ff613423e20180e3c853c6e7db57899ac6b4e327ccfd4: Status 404 returned error can't find the container with id 0316b56d9e8177cee54ff613423e20180e3c853c6e7db57899ac6b4e327ccfd4
Oct 30 21:44:33.731184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.731151    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:33.731267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.731222    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:33.731267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.731254    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:33.757769 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.757744    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-token\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.757907 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.757888    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r48gf\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-kube-api-access-r48gf\") pod \"e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") " pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:33.856430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:33.856393    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:34.174279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.174186    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" event={"ID":"0d84241f-3219-4f7c-9af6-084c34f8b986","Type":"ContainerStarted","Data":"73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb"}
Oct 30 21:44:34.174279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.174245    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" event={"ID":"0d84241f-3219-4f7c-9af6-084c34f8b986","Type":"ContainerStarted","Data":"0316b56d9e8177cee54ff613423e20180e3c853c6e7db57899ac6b4e327ccfd4"}
Oct 30 21:44:34.252376 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.252317    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:34.253763 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.253735    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:44:34.253895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.253782    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:44:34.299491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.299433    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podStartSLOduration=2.299419957 podStartE2EDuration="2.299419957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:34.296528168 +0000 UTC m=+1809.584250714" watchObservedRunningTime="2026-10-30 21:44:34.299419957 +0000 UTC m=+1809.587142503"
Oct 30 21:44:34.322744 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:44:34.322715    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fba7fc_aa17_4db1_9d8b_3ac5d0084483.slice/crio-9f96e4dbf178cbecbba162d92673e7e613ef7271c2367c6b13ef7aefe337db00 WatchSource:0}: Error finding container 9f96e4dbf178cbecbba162d92673e7e613ef7271c2367c6b13ef7aefe337db00: Status 404 returned error can't find the container with id 9f96e4dbf178cbecbba162d92673e7e613ef7271c2367c6b13ef7aefe337db00
Oct 30 21:44:34.325251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.325215    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:34.325395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.325296    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:34.325395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.325339    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:44:34.346307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:34.346281    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"]
Oct 30 21:44:35.183330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:35.183286    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" event={"ID":"29fba7fc-aa17-4db1-9d8b-3ac5d0084483","Type":"ContainerStarted","Data":"6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4"}
Oct 30 21:44:35.183330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:35.183340    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" event={"ID":"29fba7fc-aa17-4db1-9d8b-3ac5d0084483","Type":"ContainerStarted","Data":"9f96e4dbf178cbecbba162d92673e7e613ef7271c2367c6b13ef7aefe337db00"}
Oct 30 21:44:35.256017 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:35.255338    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:44:35.256017 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:35.255418    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:44:35.857145 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:35.856835    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:35.858409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:35.858325    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:44:35.858543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:35.858433    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:44:36.252566 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:36.252531    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:44:36.253089 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:36.252585    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:44:36.829372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:36.828643    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:36.850887 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:36.850858    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:36.857264 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:36.857233    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:44:36.857413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:36.857286    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:44:36.974371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:36.974288    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podStartSLOduration=3.9742739780000003 podStartE2EDuration="3.974273978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:35.306665679 +0000 UTC m=+1810.594388251" watchObservedRunningTime="2026-10-30 21:44:36.974273978 +0000 UTC m=+1812.261996525"
Oct 30 21:44:37.252022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:37.251988    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:44:37.252199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:37.252077    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:44:37.254436 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:37.254414    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:37.857654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:37.857617    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:44:37.857846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:37.857671    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:44:38.251881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:38.251848    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:44:38.252083 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:38.251897    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:44:38.856754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:38.856718    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:44:38.857235 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:38.856778    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:44:39.251894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:39.251861    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:44:39.252138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:39.251927    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:44:39.857490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:39.857397    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:44:39.857490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:39.857457    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:44:40.252000 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:40.251963    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:44:40.252198 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:40.252036    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:44:40.857808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:40.857771    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:44:40.858387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:40.857836    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:44:41.643634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:41.643595    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:41.645055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:41.645030    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:41.645710 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:41.645686    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:44:41.857237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:41.857191    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:44:41.857468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:41.857302    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:44:43.360374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:43.350060    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:43.386112 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:43.386080    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:43.440371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:43.440137    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:44:44.155771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.155712    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.156469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.156429    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.220373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.216378    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.268373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.267549    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.379523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.379470    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.404558 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.404517    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.482767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.482696    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-htr9j" podUID="86024e40-8bdb-4fcd-b5e8-1cb2045a5925" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.24:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.496504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.496424    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:44:44.531844 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.531805    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:44.600059 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:44.600015    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:44:45.030506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.030449    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.156503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.156438    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:44:45.157388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.157360    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:44:45.159644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.158176    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.160110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.160093    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:44:45.235395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.234069    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.235395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.234472    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:44:45.269469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.269426    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.270056 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.270033    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:44:45.395367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.394461    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:44:45.395367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.394884    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:45.415272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.405516    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.431746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.431270    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:44:45.497823 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.497434    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.504221 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.504196    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:44:45.532661 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.532626    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.533161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.533140    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:44:45.607812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.607770    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:44:45.608411 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.608387    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:44:45.629295 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.628654    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:44:45.640746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.640682    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:44:45.659713 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.659642    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:44:45.685268 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.685246    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:44:45.747186 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.745069    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:44:45.765701 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.762879    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:44:45.766304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.766272    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:44:45.815482 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.815447    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:44:45.825854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:44:45.825825    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:44:46.339852 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:44:46.339807    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:45:00.207858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.207823    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"]
Oct 30 21:45:00.212585 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.212557    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"]
Oct 30 21:45:00.212725 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.212672    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.216774 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.216745    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.243375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.243334    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"]
Oct 30 21:45:00.247218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.247200    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.262830 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.262814    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c6npn\""
Oct 30 21:45:00.291119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.291092    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"]
Oct 30 21:45:00.336842 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.336810    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/537f19fd-f64e-48ab-af85-0324011f483a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.336994 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.336933    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5630c16-4c9f-4b21-bac6-e71c8742f51d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.337065 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.337003    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4glns\" (UniqueName: \"kubernetes.io/projected/537f19fd-f64e-48ab-af85-0324011f483a-kube-api-access-4glns\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.337141 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.337118    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72ww6\" (UniqueName: \"kubernetes.io/projected/866f4c6c-9111-4d46-83b0-3a4847a1900d-kube-api-access-72ww6\") pod \"maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.337244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.337224    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2dvb\" (UniqueName: \"kubernetes.io/projected/d5630c16-4c9f-4b21-bac6-e71c8742f51d-kube-api-access-q2dvb\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.337873 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.337404    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866f4c6c-9111-4d46-83b0-3a4847a1900d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.347032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.347003    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"]
Oct 30 21:45:00.403948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.403923    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"]
Oct 30 21:45:00.439125 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.439087    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/537f19fd-f64e-48ab-af85-0324011f483a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.439258 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.439151    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5630c16-4c9f-4b21-bac6-e71c8742f51d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.439258 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.439207    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4glns\" (UniqueName: \"kubernetes.io/projected/537f19fd-f64e-48ab-af85-0324011f483a-kube-api-access-4glns\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.439408 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.439262    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72ww6\" (UniqueName: \"kubernetes.io/projected/866f4c6c-9111-4d46-83b0-3a4847a1900d-kube-api-access-72ww6\") pod \"maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.439408 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.439327    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2dvb\" (UniqueName: \"kubernetes.io/projected/d5630c16-4c9f-4b21-bac6-e71c8742f51d-kube-api-access-q2dvb\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.439889 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.439866    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/537f19fd-f64e-48ab-af85-0324011f483a-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.439999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.439922    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866f4c6c-9111-4d46-83b0-3a4847a1900d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.440120 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.440097    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5630c16-4c9f-4b21-bac6-e71c8742f51d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.440463 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.440444    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866f4c6c-9111-4d46-83b0-3a4847a1900d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.499199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.499140    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4glns\" (UniqueName: \"kubernetes.io/projected/537f19fd-f64e-48ab-af85-0324011f483a-kube-api-access-4glns\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.499291 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.499259    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2dvb\" (UniqueName: \"kubernetes.io/projected/d5630c16-4c9f-4b21-bac6-e71c8742f51d-kube-api-access-q2dvb\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.499834 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.499813    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72ww6\" (UniqueName: \"kubernetes.io/projected/866f4c6c-9111-4d46-83b0-3a4847a1900d-kube-api-access-72ww6\") pod \"maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.529459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.529437    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:00.536425 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.536409    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:00.559753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.559726    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:00.908874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.908794    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"]
Oct 30 21:45:00.941182 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:00.941155    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-36dc9538b2eecd1066a0d22eac241faa593adfe2e9a424572d7eb7d6b4390639 WatchSource:0}: Error finding container 36dc9538b2eecd1066a0d22eac241faa593adfe2e9a424572d7eb7d6b4390639: Status 404 returned error can't find the container with id 36dc9538b2eecd1066a0d22eac241faa593adfe2e9a424572d7eb7d6b4390639
Oct 30 21:45:00.942988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.942970    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:45:00.967238 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:00.967202    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5630c16_4c9f_4b21_bac6_e71c8742f51d.slice/crio-8f1ecc84bcfdb42d20bcb6a724a335429069a4f427a09b4c135336d6769e8827 WatchSource:0}: Error finding container 8f1ecc84bcfdb42d20bcb6a724a335429069a4f427a09b4c135336d6769e8827: Status 404 returned error can't find the container with id 8f1ecc84bcfdb42d20bcb6a724a335429069a4f427a09b4c135336d6769e8827
Oct 30 21:45:00.969129 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.969094    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"]
Oct 30 21:45:00.978883 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:00.978860    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"]
Oct 30 21:45:00.979278 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:00.979254    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537f19fd_f64e_48ab_af85_0324011f483a.slice/crio-207a36226dd31bd57b4337813cdf0ec72ed6a0140e4222630c446b1311cfdc98 WatchSource:0}: Error finding container 207a36226dd31bd57b4337813cdf0ec72ed6a0140e4222630c446b1311cfdc98: Status 404 returned error can't find the container with id 207a36226dd31bd57b4337813cdf0ec72ed6a0140e4222630c446b1311cfdc98
Oct 30 21:45:01.745243 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.745209    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"]
Oct 30 21:45:01.751072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.751046    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.855583 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.855550    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"]
Oct 30 21:45:01.861399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861261    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.861399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861322    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.861620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861500    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.861620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861579    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.861736 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861621    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adfa29b9-06f0-4a23-b49e-154decfb13db-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.861792 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861746    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.861938 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861856    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.861938 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861931    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.862083 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.861960    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7czmq\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-kube-api-access-7czmq\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.864586 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.864552    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerStarted","Data":"36dc9538b2eecd1066a0d22eac241faa593adfe2e9a424572d7eb7d6b4390639"}
Oct 30 21:45:01.868049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.868022    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" event={"ID":"537f19fd-f64e-48ab-af85-0324011f483a","Type":"ContainerStarted","Data":"207a36226dd31bd57b4337813cdf0ec72ed6a0140e4222630c446b1311cfdc98"}
Oct 30 21:45:01.869870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.869835    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerStarted","Data":"8f1ecc84bcfdb42d20bcb6a724a335429069a4f427a09b4c135336d6769e8827"}
Oct 30 21:45:01.963129 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963083    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.963428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963383    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.963549 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963527    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.963620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963588    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adfa29b9-06f0-4a23-b49e-154decfb13db-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.963681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963656    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.963855 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963834    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.963950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963860    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-certs\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.964029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.963972    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.964029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.964002    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.964029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.964013    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7czmq\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-kube-api-access-7czmq\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.964227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.964040    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-credential-socket\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.964227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.964121    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.964318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.964293    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-data\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.964787 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.964760    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adfa29b9-06f0-4a23-b49e-154decfb13db-istiod-ca-cert\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.966537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.966515    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-envoy\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:01.966790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:01.966771    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-podinfo\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:02.022672 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.022593    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-token\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:02.030458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.030428    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7czmq\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-kube-api-access-7czmq\") pod \"e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") " pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:02.068555 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.068523    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:02.547745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.547704    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"]
Oct 30 21:45:02.549253 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:02.549224    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadfa29b9_06f0_4a23_b49e_154decfb13db.slice/crio-b6ea99788eefa0050e7a7ade62387053da12f3b8be5b72697309b407a6589ecf WatchSource:0}: Error finding container b6ea99788eefa0050e7a7ade62387053da12f3b8be5b72697309b407a6589ecf: Status 404 returned error can't find the container with id b6ea99788eefa0050e7a7ade62387053da12f3b8be5b72697309b407a6589ecf
Oct 30 21:45:02.552105 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.552063    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:02.552180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.552152    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:02.552221 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.552196    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:02.878543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.878441    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" event={"ID":"adfa29b9-06f0-4a23-b49e-154decfb13db","Type":"ContainerStarted","Data":"e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9"}
Oct 30 21:45:02.878543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:02.878508    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" event={"ID":"adfa29b9-06f0-4a23-b49e-154decfb13db","Type":"ContainerStarted","Data":"b6ea99788eefa0050e7a7ade62387053da12f3b8be5b72697309b407a6589ecf"}
Oct 30 21:45:03.006960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:03.006904    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podStartSLOduration=2.006888958 podStartE2EDuration="2.006888958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:03.004048637 +0000 UTC m=+1838.291771184" watchObservedRunningTime="2026-10-30 21:45:03.006888958 +0000 UTC m=+1838.294611505"
Oct 30 21:45:03.069445 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:03.069405    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:03.071059 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:03.071029    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:45:03.071274 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:03.071247    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:45:04.069444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:04.069410    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:45:04.070164 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:04.069461    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:45:05.069831 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:05.069794    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:45:05.070304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:05.069846    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:45:06.070109 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:06.069961    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:45:06.070109 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:06.070026    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:45:06.950806 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:06.950771    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerStarted","Data":"6337863a0191062990d649120b2668c79d9b070aa71126c577a33b0e9a48efe3"}
Oct 30 21:45:06.952446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:06.952418    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerStarted","Data":"d8c98a2cfa4dad344e8038b69a358c716037de859fbd0491ddb7f83ac1f992c2"}
Oct 30 21:45:07.069490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.069458    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:45:07.069603 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.069503    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:45:07.076606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.076560    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" podStartSLOduration=1.6906071900000001 podStartE2EDuration="7.076545019s" totalImagesPullingTime="5.385937829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:07.074577165 +0000 UTC m=+1842.362299712" watchObservedRunningTime="2026-10-30 21:45:07.076545019 +0000 UTC m=+1842.364267565"
Oct 30 21:45:07.210924 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.189259    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" podStartSLOduration=2.065224268 podStartE2EDuration="7.189239869s" totalImagesPullingTime="5.124015601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:07.185596565 +0000 UTC m=+1842.473319111" watchObservedRunningTime="2026-10-30 21:45:07.189239869 +0000 UTC m=+1842.476962416"
Oct 30 21:45:07.960243 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.960202    5817 generic.go:356] "Generic (PLEG): container finished" podID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerID="d8c98a2cfa4dad344e8038b69a358c716037de859fbd0491ddb7f83ac1f992c2" exitCode=60
Oct 30 21:45:07.960453 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.960291    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerDied","Data":"d8c98a2cfa4dad344e8038b69a358c716037de859fbd0491ddb7f83ac1f992c2"}
Oct 30 21:45:07.960682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.960660    5817 scope.go:122] "RemoveContainer" containerID="d8c98a2cfa4dad344e8038b69a358c716037de859fbd0491ddb7f83ac1f992c2"
Oct 30 21:45:07.962026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.962003    5817 generic.go:356] "Generic (PLEG): container finished" podID="537f19fd-f64e-48ab-af85-0324011f483a" containerID="f9900434e912dd6e3b184cd961dce71c9bb75d7f601eded599f098191d67a1c1" exitCode=60
Oct 30 21:45:07.962111 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.962076    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" event={"ID":"537f19fd-f64e-48ab-af85-0324011f483a","Type":"ContainerDied","Data":"f9900434e912dd6e3b184cd961dce71c9bb75d7f601eded599f098191d67a1c1"}
Oct 30 21:45:07.962396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.962339    5817 scope.go:122] "RemoveContainer" containerID="f9900434e912dd6e3b184cd961dce71c9bb75d7f601eded599f098191d67a1c1"
Oct 30 21:45:07.963895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.963871    5817 generic.go:356] "Generic (PLEG): container finished" podID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerID="6337863a0191062990d649120b2668c79d9b070aa71126c577a33b0e9a48efe3" exitCode=60
Oct 30 21:45:07.964003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.963942    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerDied","Data":"6337863a0191062990d649120b2668c79d9b070aa71126c577a33b0e9a48efe3"}
Oct 30 21:45:07.964197 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:07.964178    5817 scope.go:122] "RemoveContainer" containerID="6337863a0191062990d649120b2668c79d9b070aa71126c577a33b0e9a48efe3"
Oct 30 21:45:08.069174 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:08.069147    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:45:08.069280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:08.069199    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:45:08.974602 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:08.974548    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerStarted","Data":"81d416a1a0a81133c91ce36a3ca05a5022af041d6f1f0ef8ca8a344a50dd44af"}
Oct 30 21:45:08.977141 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:08.977111    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerStarted","Data":"7ae2378c0649db2bd24a29eb7a9314231e4911072f5a4a1a34d486305859be06"}
Oct 30 21:45:08.979492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:08.979462    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" event={"ID":"537f19fd-f64e-48ab-af85-0324011f483a","Type":"ContainerStarted","Data":"2dd02bcf597510a4adb6753c57ddaece3a444b296807247e4e7d21c394770d61"}
Oct 30 21:45:09.069520 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.069490    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 21:45:09.069671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.069546    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 21:45:09.316018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.315899    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" podStartSLOduration=3.81570624 podStartE2EDuration="9.315878523s" totalImagesPullingTime="5.500172283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:09.314975222 +0000 UTC m=+1844.602697761" watchObservedRunningTime="2026-10-30 21:45:09.315878523 +0000 UTC m=+1844.603601071"
Oct 30 21:45:09.988405 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.988369    5817 generic.go:356] "Generic (PLEG): container finished" podID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerID="7ae2378c0649db2bd24a29eb7a9314231e4911072f5a4a1a34d486305859be06" exitCode=60
Oct 30 21:45:09.988832 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.988450    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerDied","Data":"7ae2378c0649db2bd24a29eb7a9314231e4911072f5a4a1a34d486305859be06"}
Oct 30 21:45:09.988832 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.988513    5817 scope.go:122] "RemoveContainer" containerID="d8c98a2cfa4dad344e8038b69a358c716037de859fbd0491ddb7f83ac1f992c2"
Oct 30 21:45:09.988832 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.988727    5817 scope.go:122] "RemoveContainer" containerID="7ae2378c0649db2bd24a29eb7a9314231e4911072f5a4a1a34d486305859be06"
Oct 30 21:45:09.989114 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:09.989088    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79_odh-ai-gateway-infra(866f4c6c-9111-4d46-83b0-3a4847a1900d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d"
Oct 30 21:45:09.990599 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.990576    5817 generic.go:356] "Generic (PLEG): container finished" podID="537f19fd-f64e-48ab-af85-0324011f483a" containerID="2dd02bcf597510a4adb6753c57ddaece3a444b296807247e4e7d21c394770d61" exitCode=60
Oct 30 21:45:09.990680 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.990655    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" event={"ID":"537f19fd-f64e-48ab-af85-0324011f483a","Type":"ContainerDied","Data":"2dd02bcf597510a4adb6753c57ddaece3a444b296807247e4e7d21c394770d61"}
Oct 30 21:45:09.990891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.990875    5817 scope.go:122] "RemoveContainer" containerID="2dd02bcf597510a4adb6753c57ddaece3a444b296807247e4e7d21c394770d61"
Oct 30 21:45:09.991169 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:09.991140    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb_odh-ai-gateway-infra(537f19fd-f64e-48ab-af85-0324011f483a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" podUID="537f19fd-f64e-48ab-af85-0324011f483a"
Oct 30 21:45:09.992591 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.992568    5817 generic.go:356] "Generic (PLEG): container finished" podID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerID="81d416a1a0a81133c91ce36a3ca05a5022af041d6f1f0ef8ca8a344a50dd44af" exitCode=60
Oct 30 21:45:09.992660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.992627    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerDied","Data":"81d416a1a0a81133c91ce36a3ca05a5022af041d6f1f0ef8ca8a344a50dd44af"}
Oct 30 21:45:09.992886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:09.992874    5817 scope.go:122] "RemoveContainer" containerID="81d416a1a0a81133c91ce36a3ca05a5022af041d6f1f0ef8ca8a344a50dd44af"
Oct 30 21:45:09.993123 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:09.993103    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp_odh-ai-gateway-infra(d5630c16-4c9f-4b21-bac6-e71c8742f51d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d"
Oct 30 21:45:10.001530 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:10.001507    5817 scope.go:122] "RemoveContainer" containerID="f9900434e912dd6e3b184cd961dce71c9bb75d7f601eded599f098191d67a1c1"
Oct 30 21:45:10.014209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:10.014193    5817 scope.go:122] "RemoveContainer" containerID="6337863a0191062990d649120b2668c79d9b070aa71126c577a33b0e9a48efe3"
Oct 30 21:45:10.074115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:10.074091    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:10.074391 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:10.074373    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:10.075239 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:10.075219    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:45:11.001704 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:11.001670    5817 scope.go:122] "RemoveContainer" containerID="81d416a1a0a81133c91ce36a3ca05a5022af041d6f1f0ef8ca8a344a50dd44af"
Oct 30 21:45:11.002143 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:11.002052    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp_odh-ai-gateway-infra(d5630c16-4c9f-4b21-bac6-e71c8742f51d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d"
Oct 30 21:45:11.003437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:11.003412    5817 scope.go:122] "RemoveContainer" containerID="7ae2378c0649db2bd24a29eb7a9314231e4911072f5a4a1a34d486305859be06"
Oct 30 21:45:11.003790 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:11.003762    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79_odh-ai-gateway-infra(866f4c6c-9111-4d46-83b0-3a4847a1900d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d"
Oct 30 21:45:11.004988 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:11.004970    5817 scope.go:122] "RemoveContainer" containerID="2dd02bcf597510a4adb6753c57ddaece3a444b296807247e4e7d21c394770d61"
Oct 30 21:45:11.005244 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:11.005227    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb_odh-ai-gateway-infra(537f19fd-f64e-48ab-af85-0324011f483a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" podUID="537f19fd-f64e-48ab-af85-0324011f483a"
Oct 30 21:45:16.341851 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:16.341810    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:45:17.273026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.272990    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"]
Oct 30 21:45:17.277263 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.277243    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.319819 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.319790    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"]
Oct 30 21:45:17.323811 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.323792    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.356061 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.356025    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.356469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.356400    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df235d02-6ff1-4fe3-8066-83027239875a-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.356469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.356444    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqxfz\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-kube-api-access-sqxfz\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.356584 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.356565    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.356645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.356591    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.356645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.356625    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.356784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.356754    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.357057 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.357038    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df235d02-6ff1-4fe3-8066-83027239875a-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.357179 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.357089    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.371138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.371108    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"]
Oct 30 21:45:17.428161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.428119    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"]
Oct 30 21:45:17.463289 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.463251    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.463520 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.463326    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df235d02-6ff1-4fe3-8066-83027239875a-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.469459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469430    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.469618 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469505    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.469618 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469537    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.469618 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469566    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df235d02-6ff1-4fe3-8066-83027239875a-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.469798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469625    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqxfz\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-kube-api-access-sqxfz\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.469798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469669    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.469798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469707    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.469798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469765    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmkpb\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-kube-api-access-xmkpb\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.469948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469842    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.469948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469870    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.469948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.469894    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.470613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.470583    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.470741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.470623    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df235d02-6ff1-4fe3-8066-83027239875a-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.470741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.470671    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.470859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.470756    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.471034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.471011    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.471100 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.471054    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.471170 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.471150    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.471608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.471222    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.471608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.471421    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.471875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.471706    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.473305 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.473281    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.473442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.473334    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df235d02-6ff1-4fe3-8066-83027239875a-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.529504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.529437    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqxfz\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-kube-api-access-sqxfz\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.532483 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.532446    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.573236 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573200    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573314    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573400    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573432    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573455    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573501    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmkpb\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-kube-api-access-xmkpb\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573557 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573555    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573582    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573616    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573656    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573749    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-data\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.573959 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573913    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istiod-ca-cert\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.574016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.573985    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-certs\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.574251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.574229    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-credential-socket\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.575915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.575897    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-podinfo\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.575979 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.575930    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-envoy\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.591192 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.591162    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:17.634071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.633855    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-token\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.638209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.638041    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmkpb\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-kube-api-access-xmkpb\") pod \"e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") " pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:17.671486 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.671434    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"]
Oct 30 21:45:17.678313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.678289    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.749202 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.749171    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"]
Oct 30 21:45:17.775990 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.775949    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qvqb\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-kube-api-access-7qvqb\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776121    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776215 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776162    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776215 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776181    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776334    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776448 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776402    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776461    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776536    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.776623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.776607    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.877736 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.877654    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.877901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.877813    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qvqb\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-kube-api-access-7qvqb\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.877984 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.877955    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.878039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878000    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.878039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878029    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.878157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878081    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.878157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878135    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.878265 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878192    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.878265 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878241    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.878478 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878448    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-data\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.879090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.878790    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-certs\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.879090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.879027    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.879272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.879139    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istiod-ca-cert\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.879476 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.879447    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-credential-socket\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.880655 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.880634    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-podinfo\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.880735 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.880676    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-envoy\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.935685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.935663    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qvqb\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-kube-api-access-7qvqb\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.936002 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.935984    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-token\") pod \"e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") " pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:17.937952 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:17.937931    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:18.001596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.001566    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:18.052835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.047253    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"]
Oct 30 21:45:18.056499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.056041    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.056499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.056141    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.056499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.056192    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.090045 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.081559    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" event={"ID":"df235d02-6ff1-4fe3-8066-83027239875a","Type":"ContainerStarted","Data":"2f261b261dce5f19705a02f345c75e758df40531e4cdcdfea89ac80e5d6c6150"}
Oct 30 21:45:18.329653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.329622    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"]
Oct 30 21:45:18.330048 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:18.330024    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fcc6c1_f818_4dd2_a7e2_b57296a962eb.slice/crio-765f5dd6b061e63e18c36177133864be240cc26e6c76ca4e1eb5004c23d16b9f WatchSource:0}: Error finding container 765f5dd6b061e63e18c36177133864be240cc26e6c76ca4e1eb5004c23d16b9f: Status 404 returned error can't find the container with id 765f5dd6b061e63e18c36177133864be240cc26e6c76ca4e1eb5004c23d16b9f
Oct 30 21:45:18.332889 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.332855    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.332981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.332921    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.332981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.332951    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.419556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.419526    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"]
Oct 30 21:45:18.421055 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:18.421023    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d09d74d_9e52_4e28_a306_f429b2bb0a65.slice/crio-64acfe0b9911ac4990483eab8250bd7e85d89781139aa437bae167af8ee1c282 WatchSource:0}: Error finding container 64acfe0b9911ac4990483eab8250bd7e85d89781139aa437bae167af8ee1c282: Status 404 returned error can't find the container with id 64acfe0b9911ac4990483eab8250bd7e85d89781139aa437bae167af8ee1c282
Oct 30 21:45:18.424966 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.424930    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.425058 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.425017    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.425058 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.425046    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:18.507868 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.507839    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"]
Oct 30 21:45:18.514128 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.514107    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.589646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.589606    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"]
Oct 30 21:45:18.697491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.697403    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-certs\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.697491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.697456    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.697741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.697686    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-credential-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.697819 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.697745    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a1e08c-32a1-420c-8187-1f631b3f7077-istiod-ca-cert\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.697819 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.697817    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-envoy\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.697913 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.697853    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-data\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.697913 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.697884    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-podinfo\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.698055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.698038    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-token\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.698144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.698125    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfg6t\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-kube-api-access-vfg6t\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.799831 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.799799    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-credential-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.799997 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.799880    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a1e08c-32a1-420c-8187-1f631b3f7077-istiod-ca-cert\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.799997 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.799945    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-envoy\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.799997 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.799986    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-data\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800031    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-podinfo\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800098    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-token\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800167    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfg6t\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-kube-api-access-vfg6t\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800224    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-certs\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800258    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800269    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-credential-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800681    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a1e08c-32a1-420c-8187-1f631b3f7077-istiod-ca-cert\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.800874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.800850    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-socket\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.801043 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.801020    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-data\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.801133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.801109    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-certs\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.802724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.802699    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-envoy\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.803125 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.803101    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-podinfo\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.865304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.865237    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfg6t\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-kube-api-access-vfg6t\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:18.865515 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:18.865411    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-token\") pod \"e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") " pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:19.114650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.114604    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" event={"ID":"10fcc6c1-f818-4dd2-a7e2-b57296a962eb","Type":"ContainerStarted","Data":"fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535"}
Oct 30 21:45:19.116088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.116059    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" event={"ID":"10fcc6c1-f818-4dd2-a7e2-b57296a962eb","Type":"ContainerStarted","Data":"765f5dd6b061e63e18c36177133864be240cc26e6c76ca4e1eb5004c23d16b9f"}
Oct 30 21:45:19.139708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.132792    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" event={"ID":"df235d02-6ff1-4fe3-8066-83027239875a","Type":"ContainerStarted","Data":"216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0"}
Oct 30 21:45:19.139708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.136927    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" event={"ID":"0d09d74d-9e52-4e28-a306-f429b2bb0a65","Type":"ContainerStarted","Data":"d0279b942fb7e050d13960e3567c5754cc9e6e835c5c38199062145b017b4fc3"}
Oct 30 21:45:19.139708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.136986    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" event={"ID":"0d09d74d-9e52-4e28-a306-f429b2bb0a65","Type":"ContainerStarted","Data":"64acfe0b9911ac4990483eab8250bd7e85d89781139aa437bae167af8ee1c282"}
Oct 30 21:45:19.139708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.139388    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:19.252127 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.252066    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podStartSLOduration=2.252049147 podStartE2EDuration="2.252049147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:19.247949054 +0000 UTC m=+1854.535671614" watchObservedRunningTime="2026-10-30 21:45:19.252049147 +0000 UTC m=+1854.539771694"
Oct 30 21:45:19.478528 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.478458    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podStartSLOduration=2.478438753 podStartE2EDuration="2.478438753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:19.47591897 +0000 UTC m=+1854.763641552" watchObservedRunningTime="2026-10-30 21:45:19.478438753 +0000 UTC m=+1854.766161304"
Oct 30 21:45:19.643314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.597411    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:19.666012 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.665536    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:19.666012 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.665617    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:19.672528 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.672464    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podStartSLOduration=2.672442604 podStartE2EDuration="2.672442604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:19.638954023 +0000 UTC m=+1854.926676580" watchObservedRunningTime="2026-10-30 21:45:19.672442604 +0000 UTC m=+1854.960165153"
Oct 30 21:45:19.673141 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:19.673115    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a1e08c_32a1_420c_8187_1f631b3f7077.slice/crio-611fd032c4df611514a0f8293d18d5b02e92aa6bf367f7fdd84417204288a7ef WatchSource:0}: Error finding container 611fd032c4df611514a0f8293d18d5b02e92aa6bf367f7fdd84417204288a7ef: Status 404 returned error can't find the container with id 611fd032c4df611514a0f8293d18d5b02e92aa6bf367f7fdd84417204288a7ef
Oct 30 21:45:19.677595 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.677172    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:19.677595 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.677252    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:19.677595 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.677298    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:19.679311 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.679287    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"]
Oct 30 21:45:19.870410 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.862711    5817 scope.go:122] "RemoveContainer" containerID="2dd02bcf597510a4adb6753c57ddaece3a444b296807247e4e7d21c394770d61"
Oct 30 21:45:19.954678 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.954645    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:19.973018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.970593    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:19.973018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:19.970660    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:20.030369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.026730    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:20.060490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.049449    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:20.060490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.049527    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:20.187567 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.187532    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" event={"ID":"94a1e08c-32a1-420c-8187-1f631b3f7077","Type":"ContainerStarted","Data":"63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01"}
Oct 30 21:45:20.187711 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.187594    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" event={"ID":"94a1e08c-32a1-420c-8187-1f631b3f7077","Type":"ContainerStarted","Data":"611fd032c4df611514a0f8293d18d5b02e92aa6bf367f7fdd84417204288a7ef"}
Oct 30 21:45:20.385870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.385814    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podStartSLOduration=2.385793177 podStartE2EDuration="2.385793177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:20.384505311 +0000 UTC m=+1855.672227891" watchObservedRunningTime="2026-10-30 21:45:20.385793177 +0000 UTC m=+1855.673515729"
Oct 30 21:45:20.593745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.593708    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:20.594289 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.594255    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:20.852372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.851507    5817 scope.go:122] "RemoveContainer" containerID="81d416a1a0a81133c91ce36a3ca05a5022af041d6f1f0ef8ca8a344a50dd44af"
Oct 30 21:45:20.946735 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.946700    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:20.946967 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:20.946938    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:21.015238 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.015118    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:21.015238 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.015183    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:21.180119 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.147146    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:21.221366 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.218591    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:21.221366 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.218651    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:21.313940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.313901    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" event={"ID":"537f19fd-f64e-48ab-af85-0324011f483a","Type":"ContainerStarted","Data":"490337b85d9b0ad40722e6cfcf96f2a0c0e06c052e5762b38a6315b87b799ed1"}
Oct 30 21:45:21.612401 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.610168    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:21.612401 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.610223    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:21.887937 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.887063    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:21.985457 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.968368    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:21.985457 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:21.968434    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:22.036372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.021329    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:22.036372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.021426    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:22.148687 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:22.148642    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537f19fd_f64e_48ab_af85_0324011f483a.slice/crio-490337b85d9b0ad40722e6cfcf96f2a0c0e06c052e5762b38a6315b87b799ed1.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:22.158371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.154454    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:22.158371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.154531    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:22.452298 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.452258    5817 generic.go:356] "Generic (PLEG): container finished" podID="537f19fd-f64e-48ab-af85-0324011f483a" containerID="490337b85d9b0ad40722e6cfcf96f2a0c0e06c052e5762b38a6315b87b799ed1" exitCode=60
Oct 30 21:45:22.470371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.452704    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" event={"ID":"537f19fd-f64e-48ab-af85-0324011f483a","Type":"ContainerDied","Data":"490337b85d9b0ad40722e6cfcf96f2a0c0e06c052e5762b38a6315b87b799ed1"}
Oct 30 21:45:22.471175 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.471144    5817 scope.go:122] "RemoveContainer" containerID="2dd02bcf597510a4adb6753c57ddaece3a444b296807247e4e7d21c394770d61"
Oct 30 21:45:22.471969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.471948    5817 scope.go:122] "RemoveContainer" containerID="490337b85d9b0ad40722e6cfcf96f2a0c0e06c052e5762b38a6315b87b799ed1"
Oct 30 21:45:22.472674 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:22.472644    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb_odh-ai-gateway-infra(537f19fd-f64e-48ab-af85-0324011f483a)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" podUID="537f19fd-f64e-48ab-af85-0324011f483a"
Oct 30 21:45:22.538370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.538306    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerStarted","Data":"014b8800c033a66311c5ee572c93242e4376af366ff8956352512b4d494cec69"}
Oct 30 21:45:22.551020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.550988    5817 scope.go:122] "RemoveContainer" containerID="014b8800c033a66311c5ee572c93242e4376af366ff8956352512b4d494cec69"
Oct 30 21:45:22.551697 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:22.551665    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp_odh-ai-gateway-infra(d5630c16-4c9f-4b21-bac6-e71c8742f51d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d"
Oct 30 21:45:22.578485 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.578407    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"]
Oct 30 21:45:22.601383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.601336    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:22.601662 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.601620    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:22.643370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.642465    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:22.643989 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.643958    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:45:22.671476 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.671419    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:22.697168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.696707    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:22.747264 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.747197    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:22.767562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.767501    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:45:22.768788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.768757    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:22.827509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.827452    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:22.902077 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.901995    5817 scope.go:122] "RemoveContainer" containerID="7ae2378c0649db2bd24a29eb7a9314231e4911072f5a4a1a34d486305859be06"
Oct 30 21:45:22.902926 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.902887    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:22.942796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.942438    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Oct 30 21:45:22.967428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:22.956291    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:23.018391 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.014551    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:23.018391 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.014619    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:23.021254 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.021225    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:23.021490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.021454    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:23.199132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.198462    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:23.199132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.198523    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:23.646437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.634743    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:23.646437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.634806    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:23.646437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.645866    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:23.646437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.646144    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:45:23.707285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.672681    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:23.707285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.676942    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:45:23.707285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.677006    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:45:23.707285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.677044    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:45:23.707285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.677090    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:45:23.707285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.701478    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:45:23.707285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.702755    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:45:23.768653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.767972    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:23.796026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.768914    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:45:23.796026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.769929    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:23.796026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.779446    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:45:23.796603 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.796575    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:45:23.796802 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.796786    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:45:23.940782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.895446    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:45:23.940782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.895719    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:45:23.940782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.912435    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:23.940782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.939149    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:23.940782 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.939208    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:23.943471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.943431    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"]
Oct 30 21:45:23.956581 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.956542    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:23.975377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:23.956968    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:45:24.030275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.030240    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:24.030654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.030607    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:24.239370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.239152    5817 generic.go:356] "Generic (PLEG): container finished" podID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerID="014b8800c033a66311c5ee572c93242e4376af366ff8956352512b4d494cec69" exitCode=60
Oct 30 21:45:24.239575 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.239465    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerDied","Data":"014b8800c033a66311c5ee572c93242e4376af366ff8956352512b4d494cec69"}
Oct 30 21:45:24.239575 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.239533    5817 scope.go:122] "RemoveContainer" containerID="81d416a1a0a81133c91ce36a3ca05a5022af041d6f1f0ef8ca8a344a50dd44af"
Oct 30 21:45:24.325509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.251749    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:24.325509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.251806    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:24.613034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.613001    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:24.624524 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.624480    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:24.649220 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.649177    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:45:24.650068 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.649408    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:45:24.735424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.675711    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:24.763371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.720574    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:25.048429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.888874    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:25.048429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.888930    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:25.048429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.889499    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:45:25.154436 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.896694    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:45:25.196445 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.196409    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:25.196694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.196669    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:25.196786 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.076674    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:25.197467 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.896773    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:45:25.197817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.155897    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:45:25.282791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:24.896750    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:45:25.298655 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.239180    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:25.298655 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.297154    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:25.605754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.605648    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:25.606040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.605990    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:25.607061 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.607037    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:25.615706 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.615667    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:25.669379 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.662740    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:25.693634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.693589    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:45:25.744409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.743776    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:25.804454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.788647    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:25.804454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.789014    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:25.804454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.789052    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:26.028848 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:25.929524    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:45:26.081720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.050545    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:26.081720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.050605    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:26.339184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.339134    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:45:26.357753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.346599    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:45:26.366408 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.348759    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/health\": context deadline exceeded"
Oct 30 21:45:26.492018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.348856    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:26.537430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.379427    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:26.640477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.585195    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:26.640477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.585265    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:26.640477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.585376    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:26.640477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.585419    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:26.714372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.684110    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:26.928087 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.870476    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:45:26.929479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.844477    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:45:26.929798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.929765    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:26.930158 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.875513    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:45:26.971409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.930970    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:45:26.998384 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.921455    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:26.998866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.998814    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:27.042829 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:26.983810    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:27.043152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.043104    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:27.113857 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.113820    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:27.114126 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.114094    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:27.343609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.337014    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:27.477420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.460630    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:27.477774 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.375996    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:27.492519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.477955    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:27.519526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.519485    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:27.527695 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.521799    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:27.551616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.529331    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:27.551616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.549248    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:27.552693 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.552653    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:27.553078 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.553056    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:45:27.553335 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.553310    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:45:27.553636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.553613    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:45:27.802436 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.735061    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:27.802436 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.735134    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:27.845635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.834581    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:27.873380 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.872667    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/health\": context deadline exceeded"
Oct 30 21:45:27.968378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.930013    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:27.980667 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.941087    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:45:27.980919 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.941121    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:45:27.982115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.981901    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/537f19fd-f64e-48ab-af85-0324011f483a-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/537f19fd-f64e-48ab-af85-0324011f483a-maas-api-service-ca\") pod \"537f19fd-f64e-48ab-af85-0324011f483a\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") "
Oct 30 21:45:27.982115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.982011    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/537f19fd-f64e-48ab-af85-0324011f483a-kube-api-access-4glns\" (UniqueName: \"kubernetes.io/projected/537f19fd-f64e-48ab-af85-0324011f483a-kube-api-access-4glns\") pod \"537f19fd-f64e-48ab-af85-0324011f483a\" (UID: \"537f19fd-f64e-48ab-af85-0324011f483a\") "
Oct 30 21:45:27.987621 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.941162    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:27.988395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.951266    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:27.989550 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.989511    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:27.989705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.987577    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/537f19fd-f64e-48ab-af85-0324011f483a-maas-api-service-ca" pod "537f19fd-f64e-48ab-af85-0324011f483a" (UID: "537f19fd-f64e-48ab-af85-0324011f483a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:27.996286 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:27.996118    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:28.004722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.004669    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerStarted","Data":"88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a"}
Oct 30 21:45:28.004838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.004786    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:45:28.019218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.019188    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:28.019480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.019438    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:28.068394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.068299    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:28.085292 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.085258    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/537f19fd-f64e-48ab-af85-0324011f483a-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:28.110336 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.108721    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/537f19fd-f64e-48ab-af85-0324011f483a-kube-api-access-4glns" pod "537f19fd-f64e-48ab-af85-0324011f483a" (UID: "537f19fd-f64e-48ab-af85-0324011f483a"). InnerVolumeSpecName "kube-api-access-4glns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:28.158371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.155852    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:28.158371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.155912    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:28.306373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.295632    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5630c16-4c9f-4b21-bac6-e71c8742f51d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5630c16-4c9f-4b21-bac6-e71c8742f51d-maas-api-service-ca\") pod \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") "
Oct 30 21:45:28.306373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.295715    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5630c16-4c9f-4b21-bac6-e71c8742f51d-kube-api-access-q2dvb\" (UniqueName: \"kubernetes.io/projected/d5630c16-4c9f-4b21-bac6-e71c8742f51d-kube-api-access-q2dvb\") pod \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\" (UID: \"d5630c16-4c9f-4b21-bac6-e71c8742f51d\") "
Oct 30 21:45:28.306373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.296568    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4glns\" (UniqueName: \"kubernetes.io/projected/537f19fd-f64e-48ab-af85-0324011f483a-kube-api-access-4glns\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:28.316966 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.316919    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5630c16-4c9f-4b21-bac6-e71c8742f51d-maas-api-service-ca" pod "d5630c16-4c9f-4b21-bac6-e71c8742f51d" (UID: "d5630c16-4c9f-4b21-bac6-e71c8742f51d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:28.336487 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.332265    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5630c16-4c9f-4b21-bac6-e71c8742f51d-kube-api-access-q2dvb" pod "d5630c16-4c9f-4b21-bac6-e71c8742f51d" (UID: "d5630c16-4c9f-4b21-bac6-e71c8742f51d"). InnerVolumeSpecName "kube-api-access-q2dvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:28.417371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.388659    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:28.417371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.388813    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:28.417371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.413831    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5630c16-4c9f-4b21-bac6-e71c8742f51d-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:28.417371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.413866    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2dvb\" (UniqueName: \"kubernetes.io/projected/d5630c16-4c9f-4b21-bac6-e71c8742f51d-kube-api-access-q2dvb\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:28.504374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.502380    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:28.639431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.615337    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:45:28.639431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.615740    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:45:28.639431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.616033    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:45:28.639431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.616286    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:45:28.685402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.679869    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:28.685402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.679923    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:28.685402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.680437    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:28.735500 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.735452    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:45:28.743471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.743418    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:28.974432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.963934    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:28.974432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.963994    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:29.000370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.996821    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:29.000370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.998567    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:45:29.000370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.998866    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:45:29.000370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:28.999119    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:45:29.028370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.003168    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:29.253576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.099896    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:45:29.255439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.253435    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:29.255635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.206587    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:29.255872 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.255849    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:29.313012 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.312971    5817 generic.go:356] "Generic (PLEG): container finished" podID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerID="88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a" exitCode=60
Oct 30 21:45:29.313288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.313263    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerDied","Data":"88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a"}
Oct 30 21:45:29.313463 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.313446    5817 scope.go:122] "RemoveContainer" containerID="7ae2378c0649db2bd24a29eb7a9314231e4911072f5a4a1a34d486305859be06"
Oct 30 21:45:29.314169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.314146    5817 scope.go:122] "RemoveContainer" containerID="88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a"
Oct 30 21:45:29.314769 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:29.314740    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79_odh-ai-gateway-infra(866f4c6c-9111-4d46-83b0-3a4847a1900d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d"
Oct 30 21:45:29.358797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.358743    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb" event={"ID":"537f19fd-f64e-48ab-af85-0324011f483a","Type":"ContainerDied","Data":"207a36226dd31bd57b4337813cdf0ec72ed6a0140e4222630c446b1311cfdc98"}
Oct 30 21:45:29.359061 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.359043    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"
Oct 30 21:45:29.399491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.399451    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"]
Oct 30 21:45:29.400140 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.400109    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp" event={"ID":"d5630c16-4c9f-4b21-bac6-e71c8742f51d","Type":"ContainerDied","Data":"8f1ecc84bcfdb42d20bcb6a724a335429069a4f427a09b4c135336d6769e8827"}
Oct 30 21:45:29.459781 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.400365    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"
Oct 30 21:45:29.511914 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.511810    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:29.512602 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.512581    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:45:29.703860 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.703816    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:29.704138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.704106    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:29.704444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:29.704419    5817 scope.go:122] "RemoveContainer" containerID="490337b85d9b0ad40722e6cfcf96f2a0c0e06c052e5762b38a6315b87b799ed1"
Oct 30 21:45:30.071371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.070900    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:30.126519 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.126474    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:30.151377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.145985    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.157496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.146023    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.157496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.146042    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.157496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.156885    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:30.157496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.156927    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:30.174371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.171893    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.230372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.205476    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"]
Oct 30 21:45:30.230789 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.230765    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889945-b76wb"]
Oct 30 21:45:30.268389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.206414    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:30.268389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.206679    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.268389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.206653    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.268389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.206698    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.268389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.259498    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:30.268389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.259551    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:30.268389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.262925    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.298745 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.298705    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:30.343680 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.339852    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"]
Oct 30 21:45:30.482484 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.482449    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:30.482748 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.482719    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:30.543638 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.527426    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:45:30.600907 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.600816    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:30.604458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.604415    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:30.708069 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.679887    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889945-lfhxp"]
Oct 30 21:45:30.978736 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.978703    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:30.978991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:30.978961    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:31.003099 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.002969    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:31.003099 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.003028    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:31.068377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.052598    5817 scope.go:122] "RemoveContainer" containerID="014b8800c033a66311c5ee572c93242e4376af366ff8956352512b4d494cec69"
Oct 30 21:45:31.068377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.053205    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:31.132249 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.131818    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:31.166395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.160195    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:31.178389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.166640    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:31.179210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.178853    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:31.204975 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.202490    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:31.204975 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.202541    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:31.281877 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.268892    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:45:31.281877 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.269266    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:45:31.460667 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.460446    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:45:31.477382 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.473617    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:45:31.524636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.505288    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="537f19fd-f64e-48ab-af85-0324011f483a" path="/var/lib/kubelet/pods/537f19fd-f64e-48ab-af85-0324011f483a/volumes"
Oct 30 21:45:31.524636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.506330    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" path="/var/lib/kubelet/pods/d5630c16-4c9f-4b21-bac6-e71c8742f51d/volumes"
Oct 30 21:45:31.524636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.507044    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:45:31.524636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.507150    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:45:31.524636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.507515    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:45:31.643857 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.643824    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:45:31.657533 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.648418    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 21:45:31.657533 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.648495    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 21:45:31.828377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.799027    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:45:31.865480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.865441    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:45:31.886334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.873994    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:45:31.886334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.878206    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:45:31.934948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.902620    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:45:31.934948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.903481    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:45:31.946371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.942190    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 21:45:31.946371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.942247    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 21:45:31.978202 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.978167    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79" event={"ID":"866f4c6c-9111-4d46-83b0-3a4847a1900d","Type":"ContainerDied","Data":"36dc9538b2eecd1066a0d22eac241faa593adfe2e9a424572d7eb7d6b4390639"}
Oct 30 21:45:31.980479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.980455    5817 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36dc9538b2eecd1066a0d22eac241faa593adfe2e9a424572d7eb7d6b4390639"
Oct 30 21:45:31.981589 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:31.981563    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:32.003377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.002045    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 21:45:32.003377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.002109    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 21:45:32.126422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.118908    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/866f4c6c-9111-4d46-83b0-3a4847a1900d-kube-api-access-72ww6\" (UniqueName: \"kubernetes.io/projected/866f4c6c-9111-4d46-83b0-3a4847a1900d-kube-api-access-72ww6\") pod \"866f4c6c-9111-4d46-83b0-3a4847a1900d\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") "
Oct 30 21:45:32.132831 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.127387    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/866f4c6c-9111-4d46-83b0-3a4847a1900d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866f4c6c-9111-4d46-83b0-3a4847a1900d-maas-api-service-ca\") pod \"866f4c6c-9111-4d46-83b0-3a4847a1900d\" (UID: \"866f4c6c-9111-4d46-83b0-3a4847a1900d\") "
Oct 30 21:45:32.151337 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.132621    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866f4c6c-9111-4d46-83b0-3a4847a1900d-maas-api-service-ca" pod "866f4c6c-9111-4d46-83b0-3a4847a1900d" (UID: "866f4c6c-9111-4d46-83b0-3a4847a1900d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:32.151530 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.151225    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:32.151666 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.151640    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:32.153189 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.152737    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/866f4c6c-9111-4d46-83b0-3a4847a1900d-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:32.169522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.167203    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/866f4c6c-9111-4d46-83b0-3a4847a1900d-kube-api-access-72ww6" pod "866f4c6c-9111-4d46-83b0-3a4847a1900d" (UID: "866f4c6c-9111-4d46-83b0-3a4847a1900d"). InnerVolumeSpecName "kube-api-access-72ww6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:32.274724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.274201    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72ww6\" (UniqueName: \"kubernetes.io/projected/866f4c6c-9111-4d46-83b0-3a4847a1900d-kube-api-access-72ww6\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:32.285624 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.285593    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"]
Oct 30 21:45:32.286079 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.286047    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy" containerID="cri-o://7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f" gracePeriod=30
Oct 30 21:45:32.652374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:32.638463    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:33.039530 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.039482    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"
Oct 30 21:45:33.049389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.048747    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:33.050846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.050822    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:33.079369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.079322    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:33.080712 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.080685    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:45:33.139227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.139189    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:33.147516 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.147480    5817 patch_prober.go:29] interesting pod/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 21:45:33.147688 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.147542    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 21:45:33.258253 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.258222    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:33.934488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.934452    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"]
Oct 30 21:45:33.941227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:33.941203    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-19dd4f-29889945-gng79"]
Oct 30 21:45:34.065699 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.065551    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:45:34.140723 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.140673    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:34.154255 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:34.154220    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:34.664788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.664757    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:34.703878 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.703847    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:45:34.845129 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.845089    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" path="/var/lib/kubelet/pods/866f4c6c-9111-4d46-83b0-3a4847a1900d/volumes"
Oct 30 21:45:34.911186 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.911153    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"]
Oct 30 21:45:34.912189 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912167    5817 cpu_manager.go:414] "containerMap: removing container" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:34.912244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912194    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:34.912244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912217    5817 cpu_manager.go:414] "containerMap: removing container" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:34.912244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912227    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:34.912385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912249    5817 cpu_manager.go:414] "containerMap: removing container" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:34.912385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912258    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:34.912385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912277    5817 cpu_manager.go:414] "containerMap: removing container" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:34.912385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912284    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:34.912385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912326    5817 cpu_manager.go:414] "containerMap: removing container" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:34.912385 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912334    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:34.912593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912408    5817 cpu_manager.go:414] "containerMap: removing container" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:34.912593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912417    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:34.912593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912581    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:34.912685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912619    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:34.912685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912633    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:34.912685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912645    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:34.912685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912657    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:34.912685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912669    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:34.912685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912682    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:34.912852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912692    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:34.912852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.912704    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:34.917027 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.916963    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:34.970053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.969681    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c6npn\""
Oct 30 21:45:34.970053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.969733    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:45:34.973190 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.973161    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"]
Oct 30 21:45:34.989103 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.989071    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:34.989248 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.989202    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:34.989502 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:34.989478    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz6lv\" (UniqueName: \"kubernetes.io/projected/752c7fd7-d9e3-4fcd-b858-61de482508ec-kube-api-access-qz6lv\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.070861 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.070827    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:45:35.090664 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.090634    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz6lv\" (UniqueName: \"kubernetes.io/projected/752c7fd7-d9e3-4fcd-b858-61de482508ec-kube-api-access-qz6lv\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.090790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.090708    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.090790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.090751    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.091532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.091507    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.093074 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.093054    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.148952 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.148927    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz6lv\" (UniqueName: \"kubernetes.io/projected/752c7fd7-d9e3-4fcd-b858-61de482508ec-kube-api-access-qz6lv\") pod \"maas-api-revoke-sub-27e86670-8bvx8\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.236203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.236172    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:35.617808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:35.617781    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"]
Oct 30 21:45:35.618850 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:35.618822    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752c7fd7_d9e3_4fcd_b858_61de482508ec.slice/crio-3ee2fd7c3bc7ffa5cf4b6296642bec370127807e3698f4436e033560dce065a4 WatchSource:0}: Error finding container 3ee2fd7c3bc7ffa5cf4b6296642bec370127807e3698f4436e033560dce065a4: Status 404 returned error can't find the container with id 3ee2fd7c3bc7ffa5cf4b6296642bec370127807e3698f4436e033560dce065a4
Oct 30 21:45:35.954963 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:35.954929    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:36.078272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:36.078240    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8" event={"ID":"752c7fd7-d9e3-4fcd-b858-61de482508ec","Type":"ContainerStarted","Data":"d057e54e028dd9f4aaae8fe29fa123bae45a852ee2571386fd7b0d5600222cc0"}
Oct 30 21:45:36.078453 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:36.078326    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8" event={"ID":"752c7fd7-d9e3-4fcd-b858-61de482508ec","Type":"ContainerStarted","Data":"3ee2fd7c3bc7ffa5cf4b6296642bec370127807e3698f4436e033560dce065a4"}
Oct 30 21:45:36.192925 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:36.192870    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8" podStartSLOduration=2.192853758 podStartE2EDuration="2.192853758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:36.191775988 +0000 UTC m=+1871.479498534" watchObservedRunningTime="2026-10-30 21:45:36.192853758 +0000 UTC m=+1871.480576307"
Oct 30 21:45:37.086739 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.086700    5817 generic.go:356] "Generic (PLEG): container finished" podID="752c7fd7-d9e3-4fcd-b858-61de482508ec" containerID="d057e54e028dd9f4aaae8fe29fa123bae45a852ee2571386fd7b0d5600222cc0" exitCode=0
Oct 30 21:45:37.086937 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.086754    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8" event={"ID":"752c7fd7-d9e3-4fcd-b858-61de482508ec","Type":"ContainerDied","Data":"d057e54e028dd9f4aaae8fe29fa123bae45a852ee2571386fd7b0d5600222cc0"}
Oct 30 21:45:37.515418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.515391    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653256    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-certs\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653309    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-credential-socket\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653392    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-kube-api-access-wk4ch\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-kube-api-access-wk4ch\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653421    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-token\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-token\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653494    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-podinfo\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653624    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istiod-ca-cert\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653665    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-envoy\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653788    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-data\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.653856    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-socket\") pod \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\" (UID: \"8b16019b-1086-4ae4-baf6-2cd27be3bcfe\") "
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.654772    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-socket" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.655204    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-certs" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:37.655517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.655382    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-credential-socket" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:37.661321 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.661298    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istiod-ca-cert" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:37.663235 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.662931    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-data" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:37.682671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.680835    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-token" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:37.705278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.705137    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-envoy" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:37.705278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.705217    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-podinfo" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:45:37.709303 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.709269    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-kube-api-access-wk4ch" pod "8b16019b-1086-4ae4-baf6-2cd27be3bcfe" (UID: "8b16019b-1086-4ae4-baf6-2cd27be3bcfe"). InnerVolumeSpecName "kube-api-access-wk4ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:37.755847 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755816    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.755847 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755846    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.756032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755856    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.756032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755865    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.756032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755874    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk4ch\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-kube-api-access-wk4ch\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.756032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755885    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.756032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755893    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.756032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755901    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:37.756032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:37.755914    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b16019b-1086-4ae4-baf6-2cd27be3bcfe-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:38.097140 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.097102    5817 generic.go:356] "Generic (PLEG): container finished" podID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerID="7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f" exitCode=0
Oct 30 21:45:38.097280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.097150    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"
Oct 30 21:45:38.097280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.097185    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" event={"ID":"8b16019b-1086-4ae4-baf6-2cd27be3bcfe","Type":"ContainerDied","Data":"7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f"}
Oct 30 21:45:38.097280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.097245    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89" event={"ID":"8b16019b-1086-4ae4-baf6-2cd27be3bcfe","Type":"ContainerDied","Data":"1ccf73817222f8027e3592607cb99e16c5a356e2fd2fb17454fcdc46619c402e"}
Oct 30 21:45:38.097280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.097269    5817 scope.go:122] "RemoveContainer" containerID="7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f"
Oct 30 21:45:38.111366 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.111328    5817 scope.go:122] "RemoveContainer" containerID="7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f"
Oct 30 21:45:38.111664 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:38.111644    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f\": container with ID starting with 7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f not found: ID does not exist" containerID="7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f"
Oct 30 21:45:38.111721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.111674    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f"} err="failed to get container status \"7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f\": rpc error: code = NotFound desc = could not find container \"7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f\": container with ID starting with 7246b4a3216fa4a7d66a568f246c8176562bbdef170cc529d83f3905ed5b0e6f not found: ID does not exist"
Oct 30 21:45:38.238078 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.238053    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:38.257311 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.257284    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"]
Oct 30 21:45:38.262755 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.262732    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-j5r89"]
Oct 30 21:45:38.365299 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.365236    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-openshift-ca\") pod \"752c7fd7-d9e3-4fcd-b858-61de482508ec\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") "
Oct 30 21:45:38.365443 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.365424    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-xks-ca\") pod \"752c7fd7-d9e3-4fcd-b858-61de482508ec\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") "
Oct 30 21:45:38.365487 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.365473    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/752c7fd7-d9e3-4fcd-b858-61de482508ec-kube-api-access-qz6lv\" (UniqueName: \"kubernetes.io/projected/752c7fd7-d9e3-4fcd-b858-61de482508ec-kube-api-access-qz6lv\") pod \"752c7fd7-d9e3-4fcd-b858-61de482508ec\" (UID: \"752c7fd7-d9e3-4fcd-b858-61de482508ec\") "
Oct 30 21:45:38.365620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.365597    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-openshift-ca" pod "752c7fd7-d9e3-4fcd-b858-61de482508ec" (UID: "752c7fd7-d9e3-4fcd-b858-61de482508ec"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:38.366131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.366111    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-openshift-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:38.367433 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.367410    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-xks-ca" pod "752c7fd7-d9e3-4fcd-b858-61de482508ec" (UID: "752c7fd7-d9e3-4fcd-b858-61de482508ec"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:45:38.367560 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.367540    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752c7fd7-d9e3-4fcd-b858-61de482508ec-kube-api-access-qz6lv" pod "752c7fd7-d9e3-4fcd-b858-61de482508ec" (UID: "752c7fd7-d9e3-4fcd-b858-61de482508ec"). InnerVolumeSpecName "kube-api-access-qz6lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:38.467224 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.467191    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/752c7fd7-d9e3-4fcd-b858-61de482508ec-maas-api-xks-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:38.467224 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.467221    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz6lv\" (UniqueName: \"kubernetes.io/projected/752c7fd7-d9e3-4fcd-b858-61de482508ec-kube-api-access-qz6lv\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:38.845313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:38.845265    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" path="/var/lib/kubelet/pods/8b16019b-1086-4ae4-baf6-2cd27be3bcfe/volumes"
Oct 30 21:45:39.106334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:39.106242    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"
Oct 30 21:45:39.106334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:39.106271    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8" event={"ID":"752c7fd7-d9e3-4fcd-b858-61de482508ec","Type":"ContainerDied","Data":"3ee2fd7c3bc7ffa5cf4b6296642bec370127807e3698f4436e033560dce065a4"}
Oct 30 21:45:39.106571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:39.106381    5817 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ee2fd7c3bc7ffa5cf4b6296642bec370127807e3698f4436e033560dce065a4"
Oct 30 21:45:44.242236 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:44.242196    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:46.337911 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:46.337876    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:45:48.543073 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.543041    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"]
Oct 30 21:45:48.543940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.543919    5817 cpu_manager.go:414] "containerMap: removing container" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:48.543999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.543941    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="537f19fd-f64e-48ab-af85-0324011f483a" containerName="cleanup"
Oct 30 21:45:48.543999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.543957    5817 cpu_manager.go:414] "containerMap: removing container" podUID="752c7fd7-d9e3-4fcd-b858-61de482508ec" containerName="revoke-keys"
Oct 30 21:45:48.543999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.543963    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="752c7fd7-d9e3-4fcd-b858-61de482508ec" containerName="revoke-keys"
Oct 30 21:45:48.543999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.543991    5817 cpu_manager.go:414] "containerMap: removing container" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:48.543999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.543996    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5630c16-4c9f-4b21-bac6-e71c8742f51d" containerName="cleanup"
Oct 30 21:45:48.544144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.544013    5817 cpu_manager.go:414] "containerMap: removing container" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:48.544144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.544018    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="866f4c6c-9111-4d46-83b0-3a4847a1900d" containerName="cleanup"
Oct 30 21:45:48.544144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.544027    5817 cpu_manager.go:414] "containerMap: removing container" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy"
Oct 30 21:45:48.544144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.544032    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy"
Oct 30 21:45:48.544144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.544126    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="752c7fd7-d9e3-4fcd-b858-61de482508ec" containerName="revoke-keys"
Oct 30 21:45:48.544144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.544140    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b16019b-1086-4ae4-baf6-2cd27be3bcfe" containerName="istio-proxy"
Oct 30 21:45:48.547629 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.547610    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.609635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.609608    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.609767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.609734    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.609767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.609763    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.609850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.609831    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.609900 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.609871    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.609939 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.609916    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.610040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.610021    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.610159 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.610139    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmxm7\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-kube-api-access-dmxm7\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.610300 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.610284    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55cf8731-65de-4bac-abdf-9c7608ed42bf-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.641840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.641808    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"]
Oct 30 21:45:48.711495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711449    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.711495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711501    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.711693 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711617    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.711693 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711672    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.711763 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711728    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.711815 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711763    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.711865 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711821    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmxm7\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-kube-api-access-dmxm7\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.711917 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711860    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-certs\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.712014 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711994    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55cf8731-65de-4bac-abdf-9c7608ed42bf-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.712097 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.711997    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-data\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.712160 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.712091    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.712220 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.712198    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.712483 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.712460    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-credential-socket\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.712562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.712545    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55cf8731-65de-4bac-abdf-9c7608ed42bf-istiod-ca-cert\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.714016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.713993    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-envoy\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.714227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.714208    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-podinfo\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.770026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.770001    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-token\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.770208 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.770189    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmxm7\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-kube-api-access-dmxm7\") pod \"e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") " pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:48.862772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:48.862690    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:49.305171 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:45:49.305142    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cf8731_65de_4bac_abdf_9c7608ed42bf.slice/crio-204a75196eaad9fba04c78ef0b4bedc9a0cb51730e774bcdc97aac4a6a9ea233 WatchSource:0}: Error finding container 204a75196eaad9fba04c78ef0b4bedc9a0cb51730e774bcdc97aac4a6a9ea233: Status 404 returned error can't find the container with id 204a75196eaad9fba04c78ef0b4bedc9a0cb51730e774bcdc97aac4a6a9ea233
Oct 30 21:45:49.305701 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:49.305676    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"]
Oct 30 21:45:49.307952 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:49.307914    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:49.308044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:49.307994    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:49.308044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:49.308035    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:45:50.217371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:50.214419    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" event={"ID":"55cf8731-65de-4bac-abdf-9c7608ed42bf","Type":"ContainerStarted","Data":"bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30"}
Oct 30 21:45:50.217371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:50.214487    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" event={"ID":"55cf8731-65de-4bac-abdf-9c7608ed42bf","Type":"ContainerStarted","Data":"204a75196eaad9fba04c78ef0b4bedc9a0cb51730e774bcdc97aac4a6a9ea233"}
Oct 30 21:45:50.437619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:50.437543    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podStartSLOduration=2.437522833 podStartE2EDuration="2.437522833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:50.381031764 +0000 UTC m=+1885.668754311" watchObservedRunningTime="2026-10-30 21:45:50.437522833 +0000 UTC m=+1885.725245379"
Oct 30 21:45:50.863648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:50.863614    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:50.865289 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:50.865263    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:45:50.865419 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:50.865309    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:45:51.035193 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:51.035156    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:51.863209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:51.863176    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:45:51.863614 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:51.863227    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:45:52.863039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:52.863010    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:45:52.863336 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:52.863305    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:45:53.127443 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:53.127338    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:53.128118 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:53.128084    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:53.863041 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:53.863006    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:45:53.863233 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:53.863083    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:45:54.258275 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:45:54.258238    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:45:54.863606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:54.863578    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:45:54.863849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:54.863825    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:45:55.863642 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:55.863608    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 21:45:55.864116 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:55.863676    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 21:45:56.868411 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:56.868380    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:56.868854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:56.868836    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:45:56.869773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:45:56.869750    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:46:02.517834 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:02.503809    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"]
Oct 30 21:46:02.518655 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:02.518613    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy" containerID="cri-o://4ae72963ce8b2cae13438657986fa37c05b78ab6684c1b6dbd9bcda0289a3882" gracePeriod=30
Oct 30 21:46:04.004371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.004031    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:04.004902 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.004384    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:46:04.204512 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.204447    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:04.205315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.205268    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:04.236503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.236461    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:04.533849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.533802    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:04.567704 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.567653    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:46:04.815806 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.815666    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:04.816566 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:04.816518    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:46:05.040388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.017074    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:46:05.040388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.017167    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:46:05.134880 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.134334    5817 patch_prober.go:29] interesting pod/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:05.134880 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.134442    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:05.135623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.135597    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:05.140461 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.140422    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:05.227371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.184698    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:46:05.227824 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.227796    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:46:05.263504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.250570    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:05.263504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.251240    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:46:05.280402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.264445    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:46:05.280402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.264822    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:46:05.280402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.265820    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:46:05.280402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.266102    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:46:05.364540 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.364487    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:05.626623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.626573    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:05.817494 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.659281    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:05.818892 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.782742    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:46:05.819155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.819139    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:46:05.819356 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.819317    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:46:05.835077 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.817492    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:46:05.835077 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.817537    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:46:05.877000 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.876893    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:46:05.950374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.923317    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:05.981411 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.980337    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:05.981411 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:05.925557    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:46:06.166480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.057369    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:46:06.209380 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.057664    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:46:06.262374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.262026    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:06.294321 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.294280    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:06.294815 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.294783    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:46:06.433095 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.433005    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:06.433683 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.433658    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:46:06.794371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.784172    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:46:06.819550 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.819500    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:46:06.823435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.819897    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:06.900835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.851657    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:46:06.928495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:06.867840    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:46:07.191405 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:07.190016    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:46:07.191405 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:07.190751    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:46:07.263760 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:07.263712    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:07.298102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:07.298052    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:07.440371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:07.433479    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:46:07.526281 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:07.526232    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:46:07.557402 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:07.543439    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:46:07.973515 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:07.973466    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:46:07.981907 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:07.975780    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:46:08.050871 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.050840    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"]
Oct 30 21:46:08.065406 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.065366    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:46:08.137070 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.044740    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:46:08.165771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.164522    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/health\": context deadline exceeded"
Oct 30 21:46:08.165771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.164857    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:46:08.185367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.027573    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:46:08.185955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.185932    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:46:08.194046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.193755    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.210272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.206921    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:08.222380 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.216882    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:08.257427 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.255264    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:46:08.257427 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.255752    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e79f61de-openshift-default-dockercfg-5grhf\""
Oct 30 21:46:08.257427 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.255831    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:46:08.313613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.312192    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"]
Oct 30 21:46:08.320171 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.320151    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:46:08.363373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.363311    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-podinfo\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.394804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.394768    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-socket\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.395080 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.395059    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f25e68-f9e0-469a-85e4-f09007267e9b-istiod-ca-cert\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.395289 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.395271    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-certs\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.398922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.378249    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:46:08.409491 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.409466    5817 generic.go:356] "Generic (PLEG): container finished" podID="ada0244e-09c2-44f7-9200-73b6641dc731" containerID="4ae72963ce8b2cae13438657986fa37c05b78ab6684c1b6dbd9bcda0289a3882" exitCode=0
Oct 30 21:46:08.411811 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.411785    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" event={"ID":"ada0244e-09c2-44f7-9200-73b6641dc731","Type":"ContainerDied","Data":"4ae72963ce8b2cae13438657986fa37c05b78ab6684c1b6dbd9bcda0289a3882"}
Oct 30 21:46:08.412226 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.412207    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:46:08.413296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.413276    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:46:08.418567 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.417096    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:46:08.424072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.422604    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:46:08.428660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.428225    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-credential-socket\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.428660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.428309    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-token\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.428660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.428487    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-envoy\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.428660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.428549    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-data\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.429462 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.428613    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx75j\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-kube-api-access-jx75j\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.451714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.451684    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535246    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-envoy\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535332    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-data\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535485    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx75j\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-kube-api-access-jx75j\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535669    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-podinfo\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535756    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-socket\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535804    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f25e68-f9e0-469a-85e4-f09007267e9b-istiod-ca-cert\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535874    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-certs\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.535959    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-credential-socket\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.536414 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.536037    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-token\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.538663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.537114    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-credential-socket\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.538663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.537274    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f25e68-f9e0-469a-85e4-f09007267e9b-istiod-ca-cert\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.538663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.537519    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-certs\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.538663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.537542    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-socket\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.538663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.538041    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-data\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.555387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.543016    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-podinfo\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.559077 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.558704    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-envoy\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.606470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.606437    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx75j\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-kube-api-access-jx75j\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.608288 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.608235    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-token\") pod \"e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") " pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.699320 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.699288    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:08.711720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.711694    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:46:08.842093 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842058    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-istio-token\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-istio-token\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.842282 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842115    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ada0244e-09c2-44f7-9200-73b6641dc731-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada0244e-09c2-44f7-9200-73b6641dc731-istio-podinfo\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.842282 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842223    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-credential-socket\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.842439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842392    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ada0244e-09c2-44f7-9200-73b6641dc731-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada0244e-09c2-44f7-9200-73b6641dc731-istiod-ca-cert\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.842523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842455    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-kube-api-access-fh277\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-kube-api-access-fh277\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.842523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842497    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-envoy\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.843691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842631    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-data\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.843691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842701    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-socket\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.843691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842708    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-credential-socket" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:08.843691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.842742    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-certs\") pod \"ada0244e-09c2-44f7-9200-73b6641dc731\" (UID: \"ada0244e-09c2-44f7-9200-73b6641dc731\") "
Oct 30 21:46:08.843691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.843487    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.844303 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.844112    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-certs" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:08.844618 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.844476    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-data" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:08.844863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.844749    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ada0244e-09c2-44f7-9200-73b6641dc731-istiod-ca-cert" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:08.845463 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.845434    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-socket" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:08.847508 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.847473    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ada0244e-09c2-44f7-9200-73b6641dc731-istio-podinfo" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:46:08.849640 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.849606    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-envoy" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:08.849746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.849652    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-istio-token" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:08.851430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.851394    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-kube-api-access-fh277" pod "ada0244e-09c2-44f7-9200-73b6641dc731" (UID: "ada0244e-09c2-44f7-9200-73b6641dc731"). InnerVolumeSpecName "kube-api-access-fh277". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:08.945462 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945429    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.945462 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945454    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.945462 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945466    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.946204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945481    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.946204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945490    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ada0244e-09c2-44f7-9200-73b6641dc731-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.946204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945498    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ada0244e-09c2-44f7-9200-73b6641dc731-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.946204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945508    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh277\" (UniqueName: \"kubernetes.io/projected/ada0244e-09c2-44f7-9200-73b6641dc731-kube-api-access-fh277\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:08.946204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:08.945517    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ada0244e-09c2-44f7-9200-73b6641dc731-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:09.140827 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:09.140794    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f25e68_f9e0_469a_85e4_f09007267e9b.slice/crio-589a01e2573d0682425c662091404ef0320f6f0187ade3c73fbb1c4b058922f8 WatchSource:0}: Error finding container 589a01e2573d0682425c662091404ef0320f6f0187ade3c73fbb1c4b058922f8: Status 404 returned error can't find the container with id 589a01e2573d0682425c662091404ef0320f6f0187ade3c73fbb1c4b058922f8
Oct 30 21:46:09.142155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.142130    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"]
Oct 30 21:46:09.143545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.143510    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:09.143619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.143577    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:09.143619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.143604    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:09.423526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.423426    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"
Oct 30 21:46:09.423526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.423477    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft" event={"ID":"ada0244e-09c2-44f7-9200-73b6641dc731","Type":"ContainerDied","Data":"8e4f7e30a1bc20ada5bc721a612154adc61894933248a6c4dac94d80a62739df"}
Oct 30 21:46:09.424003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.423550    5817 scope.go:122] "RemoveContainer" containerID="4ae72963ce8b2cae13438657986fa37c05b78ab6684c1b6dbd9bcda0289a3882"
Oct 30 21:46:09.426746 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.426698    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" event={"ID":"28f25e68-f9e0-469a-85e4-f09007267e9b","Type":"ContainerStarted","Data":"4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23"}
Oct 30 21:46:09.427921 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.427893    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" event={"ID":"28f25e68-f9e0-469a-85e4-f09007267e9b","Type":"ContainerStarted","Data":"589a01e2573d0682425c662091404ef0320f6f0187ade3c73fbb1c4b058922f8"}
Oct 30 21:46:09.699537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.699498    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:09.861297 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.861229    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podStartSLOduration=3.861213983 podStartE2EDuration="3.861213983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:09.856807131 +0000 UTC m=+1905.144529692" watchObservedRunningTime="2026-10-30 21:46:09.861213983 +0000 UTC m=+1905.148936528"
Oct 30 21:46:09.973691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.973658    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"]
Oct 30 21:46:09.976971 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:09.976946    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-n5tft"]
Oct 30 21:46:10.704535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:10.704505    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:10.845943 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:10.845908    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" path="/var/lib/kubelet/pods/ada0244e-09c2-44f7-9200-73b6641dc731/volumes"
Oct 30 21:46:11.452363 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:11.452324    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:11.453155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:11.453134    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:46:16.194271 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.194235    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"]
Oct 30 21:46:16.194975 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.194957    5817 cpu_manager.go:414] "containerMap: removing container" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy"
Oct 30 21:46:16.195026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.194977    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy"
Oct 30 21:46:16.195142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.195131    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="ada0244e-09c2-44f7-9200-73b6641dc731" containerName="istio-proxy"
Oct 30 21:46:16.198890 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.198871    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.248865 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.248843    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e79f61de-dockercfg-hpn8d\""
Oct 30 21:46:16.263990 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.263960    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e60e-3c7a-4026-93aa-9a95f675a56e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.264097 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.264078    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m78ck\" (UniqueName: \"kubernetes.io/projected/fe06e60e-3c7a-4026-93aa-9a95f675a56e-kube-api-access-m78ck\") pod \"payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.295410 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.295377    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e79f61de\""
Oct 30 21:46:16.300943 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.300916    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"]
Oct 30 21:46:16.342964 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:16.342928    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:46:16.357983 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.357943    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"]
Oct 30 21:46:16.363569 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.363540    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.365409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.365376    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m78ck\" (UniqueName: \"kubernetes.io/projected/fe06e60e-3c7a-4026-93aa-9a95f675a56e-kube-api-access-m78ck\") pod \"payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.365561 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.365545    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/260b534a-ef58-4291-a086-f3de6d7b436f-plugins-config-volume\") pod \"payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") " pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.365723 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.365693    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e60e-3c7a-4026-93aa-9a95f675a56e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.365786 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.365763    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhtf4\" (UniqueName: \"kubernetes.io/projected/260b534a-ef58-4291-a086-f3de6d7b436f-kube-api-access-jhtf4\") pod \"payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") " pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.366545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.366521    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e60e-3c7a-4026-93aa-9a95f675a56e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.426402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.420859    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"]
Oct 30 21:46:16.426402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.425304    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m78ck\" (UniqueName: \"kubernetes.io/projected/fe06e60e-3c7a-4026-93aa-9a95f675a56e-kube-api-access-m78ck\") pod \"payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.466616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.466583    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/260b534a-ef58-4291-a086-f3de6d7b436f-plugins-config-volume\") pod \"payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") " pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.466811 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.466695    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhtf4\" (UniqueName: \"kubernetes.io/projected/260b534a-ef58-4291-a086-f3de6d7b436f-kube-api-access-jhtf4\") pod \"payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") " pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.467317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.467289    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/260b534a-ef58-4291-a086-f3de6d7b436f-plugins-config-volume\") pod \"payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") " pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.514025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.513992    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:16.525267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.525237    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhtf4\" (UniqueName: \"kubernetes.io/projected/260b534a-ef58-4291-a086-f3de6d7b436f-kube-api-access-jhtf4\") pod \"payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") " pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.696965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.696920    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:16.944476 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:16.944443    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"]
Oct 30 21:46:16.944878 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:16.944854    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe06e60e_3c7a_4026_93aa_9a95f675a56e.slice/crio-37e7afafa4740f15a502d91a74bb204d1926557173e9a28e1a8747da52f3ad40 WatchSource:0}: Error finding container 37e7afafa4740f15a502d91a74bb204d1926557173e9a28e1a8747da52f3ad40: Status 404 returned error can't find the container with id 37e7afafa4740f15a502d91a74bb204d1926557173e9a28e1a8747da52f3ad40
Oct 30 21:46:17.118730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.118698    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"]
Oct 30 21:46:17.119320 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:17.119286    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260b534a_ef58_4291_a086_f3de6d7b436f.slice/crio-3469b551f053e2ad762f6909ab81ecd14310a1b6890cba38a0c76f996080954f WatchSource:0}: Error finding container 3469b551f053e2ad762f6909ab81ecd14310a1b6890cba38a0c76f996080954f: Status 404 returned error can't find the container with id 3469b551f053e2ad762f6909ab81ecd14310a1b6890cba38a0c76f996080954f
Oct 30 21:46:17.504704 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.504644    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9" event={"ID":"fe06e60e-3c7a-4026-93aa-9a95f675a56e","Type":"ContainerStarted","Data":"2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4"}
Oct 30 21:46:17.506217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.504748    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9" event={"ID":"fe06e60e-3c7a-4026-93aa-9a95f675a56e","Type":"ContainerStarted","Data":"37e7afafa4740f15a502d91a74bb204d1926557173e9a28e1a8747da52f3ad40"}
Oct 30 21:46:17.506217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.504946    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:17.507216 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.507182    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4" event={"ID":"260b534a-ef58-4291-a086-f3de6d7b436f","Type":"ContainerStarted","Data":"12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d"}
Oct 30 21:46:17.507464 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.507237    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4" event={"ID":"260b534a-ef58-4291-a086-f3de6d7b436f","Type":"ContainerStarted","Data":"3469b551f053e2ad762f6909ab81ecd14310a1b6890cba38a0c76f996080954f"}
Oct 30 21:46:17.508011 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.507947    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:17.620826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.619618    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9" podStartSLOduration=1.6196000750000001 podStartE2EDuration="1.619600075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:17.618871047 +0000 UTC m=+1912.906593624" watchObservedRunningTime="2026-10-30 21:46:17.619600075 +0000 UTC m=+1912.907322626"
Oct 30 21:46:17.672227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.672187    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"]
Oct 30 21:46:17.672714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.672679    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy" containerID="cri-o://021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7" gracePeriod=30
Oct 30 21:46:17.731139 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:17.731076    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4" podStartSLOduration=1.731057792 podStartE2EDuration="1.731057792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:17.729772453 +0000 UTC m=+1913.017494999" watchObservedRunningTime="2026-10-30 21:46:17.731057792 +0000 UTC m=+1913.018780338"
Oct 30 21:46:18.101478 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:18.101442    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:46:20.954052 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:20.954009    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f4c6c_9111_4d46_83b0_3a4847a1900d.slice/crio-conmon-88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:46:22.967926 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:22.967903    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:46:23.078985 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.078958    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-certs\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.078995    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-data\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079034    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-envoy\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079226 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079176    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-socket\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079226    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-token\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-token\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079266    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-certs" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:23.079314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079273    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-data" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:23.079463 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079312    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istiod-ca-cert\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079463 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079375    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-kube-api-access-nqv42\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-kube-api-access-nqv42\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079463 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079424    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-credential-socket\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079467    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-socket" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:23.079632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079494    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-podinfo\") pod \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\" (UID: \"0c44396d-d8fd-417b-bf4e-95b1b5256e76\") "
Oct 30 21:46:23.079753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079685    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-credential-socket" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:23.079753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.079728    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istiod-ca-cert" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:23.080282 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.080262    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.080282 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.080281    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.080483 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.080290    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.080483 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.080300    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.080483 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.080309    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.081554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.081524    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-podinfo" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:46:23.081677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.081655    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-token" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:23.081894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.081872    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-envoy" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:23.081960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.081903    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-kube-api-access-nqv42" pod "0c44396d-d8fd-417b-bf4e-95b1b5256e76" (UID: "0c44396d-d8fd-417b-bf4e-95b1b5256e76"). InnerVolumeSpecName "kube-api-access-nqv42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:23.196162 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.196081    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.196162 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.196110    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.196162 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.196122    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqv42\" (UniqueName: \"kubernetes.io/projected/0c44396d-d8fd-417b-bf4e-95b1b5256e76-kube-api-access-nqv42\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.196162 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.196131    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c44396d-d8fd-417b-bf4e-95b1b5256e76-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:23.552494 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.552460    5817 generic.go:356] "Generic (PLEG): container finished" podID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerID="021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7" exitCode=0
Oct 30 21:46:23.552650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.552523    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"
Oct 30 21:46:23.552650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.552536    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" event={"ID":"0c44396d-d8fd-417b-bf4e-95b1b5256e76","Type":"ContainerDied","Data":"021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7"}
Oct 30 21:46:23.552650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.552614    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929" event={"ID":"0c44396d-d8fd-417b-bf4e-95b1b5256e76","Type":"ContainerDied","Data":"bc023f43f6b724fa23693f58762913d14293a9fecfb8dde0239f532472fe18ac"}
Oct 30 21:46:23.552650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.552639    5817 scope.go:122] "RemoveContainer" containerID="021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7"
Oct 30 21:46:23.569454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.569430    5817 scope.go:122] "RemoveContainer" containerID="021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7"
Oct 30 21:46:23.569955 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:23.569917    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7\": container with ID starting with 021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7 not found: ID does not exist" containerID="021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7"
Oct 30 21:46:23.570050 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.569954    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7"} err="failed to get container status \"021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7\": rpc error: code = NotFound desc = could not find container \"021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7\": container with ID starting with 021ddd0f300ff849e9700e371e7d4b28cd0d939a5e70d565f750ef617a6a58e7 not found: ID does not exist"
Oct 30 21:46:23.722370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.720932    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"]
Oct 30 21:46:23.739367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.730386    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"]
Oct 30 21:46:23.739367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.731593    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy"
Oct 30 21:46:23.739367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.731615    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy"
Oct 30 21:46:23.739367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.731855    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" containerName="istio-proxy"
Oct 30 21:46:23.740394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.740004    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-lf929"]
Oct 30 21:46:23.740394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.740129    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:23.790902 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.790864    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a84c1592-dockercfg-nkd8p\""
Oct 30 21:46:23.791106 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.790915    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a84c1592\""
Oct 30 21:46:23.791397 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.791370    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"]
Oct 30 21:46:23.808428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.808297    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:23.808590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.808388    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx7pr\" (UniqueName: \"kubernetes.io/projected/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-kube-api-access-xx7pr\") pod \"payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:23.880366 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.880315    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"]
Oct 30 21:46:23.885618 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.885597    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:23.916368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.909826    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:23.916368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.910755    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx7pr\" (UniqueName: \"kubernetes.io/projected/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-kube-api-access-xx7pr\") pod \"payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:23.916368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.910685    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:24.007370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:23.994091    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx7pr\" (UniqueName: \"kubernetes.io/projected/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-kube-api-access-xx7pr\") pod \"payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:24.029421 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.009381    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"]
Oct 30 21:46:24.032130 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.029913    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44l84\" (UniqueName: \"kubernetes.io/projected/355ba04a-1220-46d0-aa21-02389ba9624c-kube-api-access-44l84\") pod \"payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") " pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:24.032130 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.030113    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355ba04a-1220-46d0-aa21-02389ba9624c-plugins-config-volume\") pod \"payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") " pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:24.055717 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.055315    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:24.131420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.131377    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44l84\" (UniqueName: \"kubernetes.io/projected/355ba04a-1220-46d0-aa21-02389ba9624c-kube-api-access-44l84\") pod \"payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") " pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:24.131597 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.131566    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355ba04a-1220-46d0-aa21-02389ba9624c-plugins-config-volume\") pod \"payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") " pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:24.132327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.132303    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355ba04a-1220-46d0-aa21-02389ba9624c-plugins-config-volume\") pod \"payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") " pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:24.190074 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.190033    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44l84\" (UniqueName: \"kubernetes.io/projected/355ba04a-1220-46d0-aa21-02389ba9624c-kube-api-access-44l84\") pod \"payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") " pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:24.201752 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.201727    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:24.455381 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:24.455307    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4df0b1_b87c_42a7_8d99_db1bd7dde573.slice/crio-546e010258be181832c0671f693bf3b1506a3373bda2c0e36b72d020f47b1f37 WatchSource:0}: Error finding container 546e010258be181832c0671f693bf3b1506a3373bda2c0e36b72d020f47b1f37: Status 404 returned error can't find the container with id 546e010258be181832c0671f693bf3b1506a3373bda2c0e36b72d020f47b1f37
Oct 30 21:46:24.456012 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.455938    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"]
Oct 30 21:46:24.561838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.561803    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9" event={"ID":"6f4df0b1-b87c-42a7-8d99-db1bd7dde573","Type":"ContainerStarted","Data":"ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040"}
Oct 30 21:46:24.561982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.561870    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:24.561982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.561897    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9" event={"ID":"6f4df0b1-b87c-42a7-8d99-db1bd7dde573","Type":"ContainerStarted","Data":"546e010258be181832c0671f693bf3b1506a3373bda2c0e36b72d020f47b1f37"}
Oct 30 21:46:24.600759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.600728    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"]
Oct 30 21:46:24.602849 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:24.602823    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355ba04a_1220_46d0_aa21_02389ba9624c.slice/crio-076d4846175c08e0de149de910fa25e319a638e6342bba0940338a6f39909258 WatchSource:0}: Error finding container 076d4846175c08e0de149de910fa25e319a638e6342bba0940338a6f39909258: Status 404 returned error can't find the container with id 076d4846175c08e0de149de910fa25e319a638e6342bba0940338a6f39909258
Oct 30 21:46:24.698387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.698326    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9" podStartSLOduration=1.698312516 podStartE2EDuration="1.698312516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:24.695937802 +0000 UTC m=+1919.983660351" watchObservedRunningTime="2026-10-30 21:46:24.698312516 +0000 UTC m=+1919.986035062"
Oct 30 21:46:24.857454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:24.857341    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c44396d-d8fd-417b-bf4e-95b1b5256e76" path="/var/lib/kubelet/pods/0c44396d-d8fd-417b-bf4e-95b1b5256e76/volumes"
Oct 30 21:46:25.571241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:25.571205    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp" event={"ID":"355ba04a-1220-46d0-aa21-02389ba9624c","Type":"ContainerStarted","Data":"8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0"}
Oct 30 21:46:25.571685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:25.571295    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:25.571685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:25.571321    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp" event={"ID":"355ba04a-1220-46d0-aa21-02389ba9624c","Type":"ContainerStarted","Data":"076d4846175c08e0de149de910fa25e319a638e6342bba0940338a6f39909258"}
Oct 30 21:46:25.687084 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:25.687025    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp" podStartSLOduration=2.6870080119999997 podStartE2EDuration="2.687008012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:25.686242486 +0000 UTC m=+1920.973965034" watchObservedRunningTime="2026-10-30 21:46:25.687008012 +0000 UTC m=+1920.974730559"
Oct 30 21:46:26.352237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.352200    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"]
Oct 30 21:46:26.357438 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.357416    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:26.410554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.410511    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"]
Oct 30 21:46:26.459368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.458585    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-899a6bb1-dockercfg-4t99d\""
Oct 30 21:46:26.459368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.458707    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-899a6bb1\""
Oct 30 21:46:26.477705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.477667    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19879e76-bbb7-41b9-a954-71c604c0351d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:26.478594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.478555    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbdnz\" (UniqueName: \"kubernetes.io/projected/19879e76-bbb7-41b9-a954-71c604c0351d-kube-api-access-hbdnz\") pod \"payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:26.529638 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.529601    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"]
Oct 30 21:46:26.537773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.537735    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:26.567555 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.567525    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"]
Oct 30 21:46:26.615779 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.614834    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19879e76-bbb7-41b9-a954-71c604c0351d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:26.617401 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.617319    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbdnz\" (UniqueName: \"kubernetes.io/projected/19879e76-bbb7-41b9-a954-71c604c0351d-kube-api-access-hbdnz\") pod \"payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:26.618144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.617187    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19879e76-bbb7-41b9-a954-71c604c0351d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:26.681443 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.681408    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbdnz\" (UniqueName: \"kubernetes.io/projected/19879e76-bbb7-41b9-a954-71c604c0351d-kube-api-access-hbdnz\") pod \"payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:26.719796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.719769    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15891565-0c25-4a12-acff-68f6aad5dfc3-plugins-config-volume\") pod \"payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") " pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:26.720252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.720222    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw7vx\" (UniqueName: \"kubernetes.io/projected/15891565-0c25-4a12-acff-68f6aad5dfc3-kube-api-access-zw7vx\") pod \"payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") " pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:26.821576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.821544    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw7vx\" (UniqueName: \"kubernetes.io/projected/15891565-0c25-4a12-acff-68f6aad5dfc3-kube-api-access-zw7vx\") pod \"payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") " pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:26.821706 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.821632    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15891565-0c25-4a12-acff-68f6aad5dfc3-plugins-config-volume\") pod \"payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") " pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:26.822303 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.822283    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15891565-0c25-4a12-acff-68f6aad5dfc3-plugins-config-volume\") pod \"payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") " pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:26.880710 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.880641    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw7vx\" (UniqueName: \"kubernetes.io/projected/15891565-0c25-4a12-acff-68f6aad5dfc3-kube-api-access-zw7vx\") pod \"payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") " pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:26.972533 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:26.972504    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:27.170399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.170280    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:27.349438 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.349408    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"]
Oct 30 21:46:27.350053 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:27.350024    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19879e76_bbb7_41b9_a954_71c604c0351d.slice/crio-3f8315d7271e2fb2d15ea47e08835694af712f68dabb4919ddf57bb4a40ef46d WatchSource:0}: Error finding container 3f8315d7271e2fb2d15ea47e08835694af712f68dabb4919ddf57bb4a40ef46d: Status 404 returned error can't find the container with id 3f8315d7271e2fb2d15ea47e08835694af712f68dabb4919ddf57bb4a40ef46d
Oct 30 21:46:27.573355 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.573315    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"]
Oct 30 21:46:27.573998 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:27.573972    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15891565_0c25_4a12_acff_68f6aad5dfc3.slice/crio-f931bdfcfde8e0996656719e307ad05eaa3b280e694b7847c999ded9de59c55c WatchSource:0}: Error finding container f931bdfcfde8e0996656719e307ad05eaa3b280e694b7847c999ded9de59c55c: Status 404 returned error can't find the container with id f931bdfcfde8e0996656719e307ad05eaa3b280e694b7847c999ded9de59c55c
Oct 30 21:46:27.600490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.600454    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz" event={"ID":"15891565-0c25-4a12-acff-68f6aad5dfc3","Type":"ContainerStarted","Data":"f931bdfcfde8e0996656719e307ad05eaa3b280e694b7847c999ded9de59c55c"}
Oct 30 21:46:27.602019 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.601992    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x" event={"ID":"19879e76-bbb7-41b9-a954-71c604c0351d","Type":"ContainerStarted","Data":"7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240"}
Oct 30 21:46:27.602111 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.602046    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:27.602111 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.602068    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x" event={"ID":"19879e76-bbb7-41b9-a954-71c604c0351d","Type":"ContainerStarted","Data":"3f8315d7271e2fb2d15ea47e08835694af712f68dabb4919ddf57bb4a40ef46d"}
Oct 30 21:46:27.723947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:27.723894    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x" podStartSLOduration=1.723880763 podStartE2EDuration="1.723880763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:27.721007472 +0000 UTC m=+1923.008730019" watchObservedRunningTime="2026-10-30 21:46:27.723880763 +0000 UTC m=+1923.011603308"
Oct 30 21:46:28.517960 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:28.517932    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:46:28.611251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:28.611218    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz" event={"ID":"15891565-0c25-4a12-acff-68f6aad5dfc3","Type":"ContainerStarted","Data":"e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07"}
Oct 30 21:46:28.611425 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:28.611298    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:28.748360 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:28.748284    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz" podStartSLOduration=2.7482702530000003 podStartE2EDuration="2.748270253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:28.744133481 +0000 UTC m=+1924.031856028" watchObservedRunningTime="2026-10-30 21:46:28.748270253 +0000 UTC m=+1924.035992800"
Oct 30 21:46:29.527132 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:29.527104    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:46:35.574999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:35.574966    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:46:36.323243 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.323208    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"]
Oct 30 21:46:36.394367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.384568    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.463117 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.462745    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-af87dbaf-openshift-default-dockercfg-lhq9c\""
Oct 30 21:46:36.466317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.465705    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"]
Oct 30 21:46:36.467857 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.467819    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-socket\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.467956 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.467887    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc664\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-kube-api-access-zc664\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.467956 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.467911    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-envoy\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.467956 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.467931    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca29db-8554-4880-877a-78576d6e0b78-istiod-ca-cert\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.468060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.468003    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-credential-socket\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.468060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.468019    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-istio-token\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.468060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.468040    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca29db-8554-4880-877a-78576d6e0b78-istio-podinfo\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.468182 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.468072    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-data\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.468182 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.468101    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-certs\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.569631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.569598    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc664\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-kube-api-access-zc664\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.569822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.569646    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-envoy\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.569822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.569674    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca29db-8554-4880-877a-78576d6e0b78-istiod-ca-cert\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.569822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.569780    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-credential-socket\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.569822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.569803    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-istio-token\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570001    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca29db-8554-4880-877a-78576d6e0b78-istio-podinfo\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570108 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570065    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-data\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570162 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570120    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-certs\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570208 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570182    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-credential-socket\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570401 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570376    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-socket\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570502 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570408    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-data\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570526    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-certs\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570565    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca29db-8554-4880-877a-78576d6e0b78-istiod-ca-cert\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.570648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.570632    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-socket\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.572554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.572530    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-envoy\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.572888 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.572855    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca29db-8554-4880-877a-78576d6e0b78-istio-podinfo\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.596241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.596174    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:46:36.628912 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.628883    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc664\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-kube-api-access-zc664\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.629065 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.628914    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-istio-token\") pod \"e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") " pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:36.772193 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:36.770685    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:37.220889 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:37.220863    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"]
Oct 30 21:46:37.222614 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:37.222590    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ca29db_8554_4880_877a_78576d6e0b78.slice/crio-a58b39aa02dfc894a0a36dee55ba832074a2c9bc861af0cea512d2434a2e77d7 WatchSource:0}: Error finding container a58b39aa02dfc894a0a36dee55ba832074a2c9bc861af0cea512d2434a2e77d7: Status 404 returned error can't find the container with id a58b39aa02dfc894a0a36dee55ba832074a2c9bc861af0cea512d2434a2e77d7
Oct 30 21:46:37.225388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:37.225336    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:37.225468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:37.225424    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:37.225468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:37.225458    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:46:37.687128 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:37.687025    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" event={"ID":"81ca29db-8554-4880-877a-78576d6e0b78","Type":"ContainerStarted","Data":"1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c"}
Oct 30 21:46:37.687128 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:37.687088    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" event={"ID":"81ca29db-8554-4880-877a-78576d6e0b78","Type":"ContainerStarted","Data":"a58b39aa02dfc894a0a36dee55ba832074a2c9bc861af0cea512d2434a2e77d7"}
Oct 30 21:46:37.771490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:37.771439    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:38.675599 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:38.660515    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:46:38.799372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:38.792651    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podStartSLOduration=2.792631473 podStartE2EDuration="2.792631473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:37.905247701 +0000 UTC m=+1933.192970249" watchObservedRunningTime="2026-10-30 21:46:38.792631473 +0000 UTC m=+1934.080354020"
Oct 30 21:46:39.203415 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:39.124025    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:39.732397 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:39.709694    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:46:39.871522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:39.871426    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:39.900664 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:39.900633    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:46:39.909732 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:39.909692    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:40.144430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.126142    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Oct 30 21:46:40.174440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.163587    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:46:40.174440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.167714    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": context deadline exceeded"
Oct 30 21:46:40.174440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.167792    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:46:40.174440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.170034    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/health\": context deadline exceeded"
Oct 30 21:46:40.227374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.226811    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:40.227374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.227184    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:40.269516 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.269474    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:40.320697 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.320654    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:40.321252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.321218    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:40.332673 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.332618    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:46:40.333267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.333230    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:46:40.840585 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.810754    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/health\": context deadline exceeded"
Oct 30 21:46:40.948542 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:40.944628    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:46:41.163552 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.163435    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:41.172374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.164131    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:41.246523 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.176423    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:41.246772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.176466    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:41.247339 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.244996    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:41.247780 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.245052    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": context deadline exceeded"
Oct 30 21:46:41.247928 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.245072    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": context deadline exceeded"
Oct 30 21:46:41.272421 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.270613    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:46:41.387508 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.387457    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.387960    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.404578    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.405514    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429476    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429522    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429572    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429610    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429718    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429753    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429788    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429824    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:46:41.499644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.429859    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:46:41.971426 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:41.959440    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:46:42.039372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.035911    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:42.049368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.045498    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:42.215370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.215115    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:42.250417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.249480    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:46:42.265368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.249431    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:42.290382 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.275284    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:46:42.290382 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.276404    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": context deadline exceeded"
Oct 30 21:46:42.290382 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.275231    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:42.448340 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.448287    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:42.473895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.473840    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:42.476446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.476403    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:46:42.556171 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.556091    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 21:46:42.601645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.601608    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:46:42.623363 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.610549    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 21:46:42.695878 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.695762    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:46:42.695878 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.695819    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:46:42.699647 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.697902    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:46:42.722936 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.722902    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 21:46:42.723108 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.722961    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:46:42.764895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.764866    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:46:42.786974 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.786949    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 21:46:42.799330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.799268    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 21:46:42.803157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:42.803106    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 21:46:45.297054 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.297018    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"]
Oct 30 21:46:45.301811 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.301789    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.351972 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.351939    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-af87dbaf-dockercfg-bvsff\""
Oct 30 21:46:45.352413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.352336    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-af87dbaf\""
Oct 30 21:46:45.356048 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.356020    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"]
Oct 30 21:46:45.356307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.356281    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8q2b\" (UniqueName: \"kubernetes.io/projected/6c75a704-fcbe-48f8-811c-0a64c8193acf-kube-api-access-p8q2b\") pod \"payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.356576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.356553    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c75a704-fcbe-48f8-811c-0a64c8193acf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.453169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.453140    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"]
Oct 30 21:46:45.456467 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.456438    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:45.457710 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.457683    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8q2b\" (UniqueName: \"kubernetes.io/projected/6c75a704-fcbe-48f8-811c-0a64c8193acf-kube-api-access-p8q2b\") pod \"payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.457818 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.457797    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c75a704-fcbe-48f8-811c-0a64c8193acf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.458574 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.458550    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c75a704-fcbe-48f8-811c-0a64c8193acf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.515874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.515844    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"]
Oct 30 21:46:45.520273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.520238    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8q2b\" (UniqueName: \"kubernetes.io/projected/6c75a704-fcbe-48f8-811c-0a64c8193acf-kube-api-access-p8q2b\") pod \"payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.559330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.559236    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb8xb\" (UniqueName: \"kubernetes.io/projected/7db130db-5113-40f1-a7de-4f4c2bf45b56-kube-api-access-bb8xb\") pod \"payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") " pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:45.559330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.559296    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db130db-5113-40f1-a7de-4f4c2bf45b56-plugins-config-volume\") pod \"payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") " pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:45.618137 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.618102    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:45.661707 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.661673    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb8xb\" (UniqueName: \"kubernetes.io/projected/7db130db-5113-40f1-a7de-4f4c2bf45b56-kube-api-access-bb8xb\") pod \"payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") " pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:45.661954 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.661934    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db130db-5113-40f1-a7de-4f4c2bf45b56-plugins-config-volume\") pod \"payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") " pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:45.663324 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.663302    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db130db-5113-40f1-a7de-4f4c2bf45b56-plugins-config-volume\") pod \"payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") " pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:45.723923 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.723892    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb8xb\" (UniqueName: \"kubernetes.io/projected/7db130db-5113-40f1-a7de-4f4c2bf45b56-kube-api-access-bb8xb\") pod \"payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") " pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:45.776925 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:45.776668    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:46.065795 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.065764    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"]
Oct 30 21:46:46.067217 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:46.067194    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c75a704_fcbe_48f8_811c_0a64c8193acf.slice/crio-fb83ee2cf1b9a9934cbd4a5b556c48e349d32456bdb308b0fddaf821d8f0f5ee WatchSource:0}: Error finding container fb83ee2cf1b9a9934cbd4a5b556c48e349d32456bdb308b0fddaf821d8f0f5ee: Status 404 returned error can't find the container with id fb83ee2cf1b9a9934cbd4a5b556c48e349d32456bdb308b0fddaf821d8f0f5ee
Oct 30 21:46:46.237388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.237362    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"]
Oct 30 21:46:46.238499 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:46.238477    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db130db_5113_40f1_a7de_4f4c2bf45b56.slice/crio-2ff4a24468bfe1454f9e569c71c8ea357b8d05253f51c961afa0b7c5e15533ed WatchSource:0}: Error finding container 2ff4a24468bfe1454f9e569c71c8ea357b8d05253f51c961afa0b7c5e15533ed: Status 404 returned error can't find the container with id 2ff4a24468bfe1454f9e569c71c8ea357b8d05253f51c961afa0b7c5e15533ed
Oct 30 21:46:46.339079 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:46.338945    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:46:46.671166 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.671069    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m" event={"ID":"7db130db-5113-40f1-a7de-4f4c2bf45b56","Type":"ContainerStarted","Data":"ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36"}
Oct 30 21:46:46.671328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.671168    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m" event={"ID":"7db130db-5113-40f1-a7de-4f4c2bf45b56","Type":"ContainerStarted","Data":"2ff4a24468bfe1454f9e569c71c8ea357b8d05253f51c961afa0b7c5e15533ed"}
Oct 30 21:46:46.671328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.671287    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:46.673053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.673021    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm" event={"ID":"6c75a704-fcbe-48f8-811c-0a64c8193acf","Type":"ContainerStarted","Data":"4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3"}
Oct 30 21:46:46.673169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.673075    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:46:46.673169 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.673096    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm" event={"ID":"6c75a704-fcbe-48f8-811c-0a64c8193acf","Type":"ContainerStarted","Data":"fb83ee2cf1b9a9934cbd4a5b556c48e349d32456bdb308b0fddaf821d8f0f5ee"}
Oct 30 21:46:46.784864 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.784817    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m" podStartSLOduration=1.784803873 podStartE2EDuration="1.784803873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:46.782029366 +0000 UTC m=+1942.069751908" watchObservedRunningTime="2026-10-30 21:46:46.784803873 +0000 UTC m=+1942.072526443"
Oct 30 21:46:46.893887 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:46.893837    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm" podStartSLOduration=1.8938236389999998 podStartE2EDuration="1.893823639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:46.89187807 +0000 UTC m=+1942.179600616" watchObservedRunningTime="2026-10-30 21:46:46.893823639 +0000 UTC m=+1942.181546183"
Oct 30 21:46:53.382787 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.382755    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"]
Oct 30 21:46:53.387027 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.387009    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.439108 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.439066    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e79f61de-serving-cert\""
Oct 30 21:46:53.486035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.485161    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-metrics-certs\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.486035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.485297    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36530fcb-a719-49b1-ba34-c90c63537d68-openshift-service-ca\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.486035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.485499    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txk9g\" (UniqueName: \"kubernetes.io/projected/36530fcb-a719-49b1-ba34-c90c63537d68-kube-api-access-txk9g\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.486035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.485589    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.489641 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.489617    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"]
Oct 30 21:46:53.586938 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.586905    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.587092 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.586991    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-metrics-certs\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.587092 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:53.587037    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-serving-cert: secret "maas-api-e2e-ait-e79f61de-serving-cert" not found
Oct 30 21:46:53.587092 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.587067    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36530fcb-a719-49b1-ba34-c90c63537d68-openshift-service-ca\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.587239 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:46:53.587124    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls podName:36530fcb-a719-49b1-ba34-c90c63537d68 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:54.087103964 +0000 UTC m=+1949.374826504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls") pod "maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6" (UID: "36530fcb-a719-49b1-ba34-c90c63537d68") : secret "maas-api-e2e-ait-e79f61de-serving-cert" not found
Oct 30 21:46:53.587358 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.587329    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txk9g\" (UniqueName: \"kubernetes.io/projected/36530fcb-a719-49b1-ba34-c90c63537d68-kube-api-access-txk9g\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.587885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.587863    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36530fcb-a719-49b1-ba34-c90c63537d68-openshift-service-ca\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.589424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.589405    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-metrics-certs\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:53.645925 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:53.645847    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txk9g\" (UniqueName: \"kubernetes.io/projected/36530fcb-a719-49b1-ba34-c90c63537d68-kube-api-access-txk9g\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:54.095753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:54.095714    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:54.098152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:54.098128    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls\") pod \"maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:54.300997 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:54.300969    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:54.675099 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:54.675071    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"]
Oct 30 21:46:54.676099 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:46:54.676073    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36530fcb_a719_49b1_ba34_c90c63537d68.slice/crio-964c35c6d2bd1120b23067e68792d0266ecb92bd25fce8da7d8c80fcb4c37a7e WatchSource:0}: Error finding container 964c35c6d2bd1120b23067e68792d0266ecb92bd25fce8da7d8c80fcb4c37a7e: Status 404 returned error can't find the container with id 964c35c6d2bd1120b23067e68792d0266ecb92bd25fce8da7d8c80fcb4c37a7e
Oct 30 21:46:54.735327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:54.735301    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6" event={"ID":"36530fcb-a719-49b1-ba34-c90c63537d68","Type":"ContainerStarted","Data":"964c35c6d2bd1120b23067e68792d0266ecb92bd25fce8da7d8c80fcb4c37a7e"}
Oct 30 21:46:55.743857 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:55.743813    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6" event={"ID":"36530fcb-a719-49b1-ba34-c90c63537d68","Type":"ContainerStarted","Data":"c1a96c20f88345383bb0376a21df5aeb56705beb7c018d338c646702490327d1"}
Oct 30 21:46:55.744236 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:55.743881    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:46:55.865145 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:55.865054    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6" podStartSLOduration=2.036458873 podStartE2EDuration="2.865038818s" totalImagesPullingTime="828.579945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:55.860756882 +0000 UTC m=+1951.148479421" watchObservedRunningTime="2026-10-30 21:46:55.865038818 +0000 UTC m=+1951.152761364"
Oct 30 21:46:57.683461 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:57.683420    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:46:57.683861 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:46:57.683510    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:47:01.757382 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:01.757339    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:47:03.038440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.038400    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"]
Oct 30 21:47:03.044759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.044731    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.145085 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.145051    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"]
Oct 30 21:47:03.147462 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.147433    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p2gs\" (UniqueName: \"kubernetes.io/projected/57d49345-36ce-4dae-b373-e78926d7ba49-kube-api-access-8p2gs\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.147625 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.147609    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d49345-36ce-4dae-b373-e78926d7ba49-openshift-service-ca\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.147719 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.147704    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-metrics-certs\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.147835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.147822    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.188135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.188101    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a84c1592-serving-cert\""
Oct 30 21:47:03.249260 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.249222    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p2gs\" (UniqueName: \"kubernetes.io/projected/57d49345-36ce-4dae-b373-e78926d7ba49-kube-api-access-8p2gs\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.249452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.249333    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d49345-36ce-4dae-b373-e78926d7ba49-openshift-service-ca\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.249452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.249442    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-metrics-certs\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.249587 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.249574    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.249773 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:47:03.249750    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-serving-cert: secret "maas-api-e2e-ait-a84c1592-serving-cert" not found
Oct 30 21:47:03.249847 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:47:03.249829    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls podName:57d49345-36ce-4dae-b373-e78926d7ba49 nodeName:}" failed. No retries permitted until 2026-10-30 21:47:03.749809457 +0000 UTC m=+1959.037531987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls") pod "maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9" (UID: "57d49345-36ce-4dae-b373-e78926d7ba49") : secret "maas-api-e2e-ait-a84c1592-serving-cert" not found
Oct 30 21:47:03.250063 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.250044    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d49345-36ce-4dae-b373-e78926d7ba49-openshift-service-ca\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.252020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.251998    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-metrics-certs\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.310786 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.310700    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p2gs\" (UniqueName: \"kubernetes.io/projected/57d49345-36ce-4dae-b373-e78926d7ba49-kube-api-access-8p2gs\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.757460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.757425    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.759863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.759839    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls\") pod \"maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:03.960064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:03.960023    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:04.347999 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:47:04.347963    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d49345_36ce_4dae_b373_e78926d7ba49.slice/crio-c99148c129c72056acaaa0139f0af6a02c5d4ffde89fde9297d620edb7c3222b WatchSource:0}: Error finding container c99148c129c72056acaaa0139f0af6a02c5d4ffde89fde9297d620edb7c3222b: Status 404 returned error can't find the container with id c99148c129c72056acaaa0139f0af6a02c5d4ffde89fde9297d620edb7c3222b
Oct 30 21:47:04.348489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:04.348244    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"]
Oct 30 21:47:04.812867 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:04.812828    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9" event={"ID":"57d49345-36ce-4dae-b373-e78926d7ba49","Type":"ContainerStarted","Data":"c99148c129c72056acaaa0139f0af6a02c5d4ffde89fde9297d620edb7c3222b"}
Oct 30 21:47:05.822722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:05.822684    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9" event={"ID":"57d49345-36ce-4dae-b373-e78926d7ba49","Type":"ContainerStarted","Data":"172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04"}
Oct 30 21:47:05.823157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:05.822756    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:05.943102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:05.943049    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9" podStartSLOduration=3.055737731 podStartE2EDuration="3.943034488s" totalImagesPullingTime="887.296757ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:05.940030926 +0000 UTC m=+1961.227753472" watchObservedRunningTime="2026-10-30 21:47:05.943034488 +0000 UTC m=+1961.230757033"
Oct 30 21:47:11.839889 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:11.839859    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:47:12.828135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.828102    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"]
Oct 30 21:47:12.833946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.833924    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:12.888374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.888327    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"]
Oct 30 21:47:12.913402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.913370    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-metrics-certs\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:12.913634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.913615    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:12.913714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.913650    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45459602-ee7f-4e32-8f09-41106c0275a5-openshift-service-ca\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:12.913796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.913780    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szx9h\" (UniqueName: \"kubernetes.io/projected/45459602-ee7f-4e32-8f09-41106c0275a5-kube-api-access-szx9h\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:12.930038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:12.930011    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-899a6bb1-serving-cert\""
Oct 30 21:47:13.015087 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.015045    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szx9h\" (UniqueName: \"kubernetes.io/projected/45459602-ee7f-4e32-8f09-41106c0275a5-kube-api-access-szx9h\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.015266 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.015213    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-metrics-certs\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.015434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.015416    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.015561 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.015456    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45459602-ee7f-4e32-8f09-41106c0275a5-openshift-service-ca\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.015561 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:47:13.015552    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-serving-cert: secret "maas-api-e2e-derive-899a6bb1-serving-cert" not found
Oct 30 21:47:13.015645 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:47:13.015617    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls podName:45459602-ee7f-4e32-8f09-41106c0275a5 nodeName:}" failed. No retries permitted until 2026-10-30 21:47:13.515600784 +0000 UTC m=+1968.803323308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls") pod "maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s" (UID: "45459602-ee7f-4e32-8f09-41106c0275a5") : secret "maas-api-e2e-derive-899a6bb1-serving-cert" not found
Oct 30 21:47:13.016158 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.016138    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45459602-ee7f-4e32-8f09-41106c0275a5-openshift-service-ca\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.017654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.017624    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-metrics-certs\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.073588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.073559    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szx9h\" (UniqueName: \"kubernetes.io/projected/45459602-ee7f-4e32-8f09-41106c0275a5-kube-api-access-szx9h\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.524594 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.524563    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.526941 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.526913    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls\") pod \"maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:13.749630 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:13.749597    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:14.144585 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:47:14.144550    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45459602_ee7f_4e32_8f09_41106c0275a5.slice/crio-7826ddcf468f0bfcef18e8b3626a05632049bc3a70b07a671fac67e4a3b59833 WatchSource:0}: Error finding container 7826ddcf468f0bfcef18e8b3626a05632049bc3a70b07a671fac67e4a3b59833: Status 404 returned error can't find the container with id 7826ddcf468f0bfcef18e8b3626a05632049bc3a70b07a671fac67e4a3b59833
Oct 30 21:47:14.145198 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:14.145170    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"]
Oct 30 21:47:14.913366 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:14.913196    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s" event={"ID":"45459602-ee7f-4e32-8f09-41106c0275a5","Type":"ContainerStarted","Data":"7826ddcf468f0bfcef18e8b3626a05632049bc3a70b07a671fac67e4a3b59833"}
Oct 30 21:47:15.978930 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:15.978889    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s" event={"ID":"45459602-ee7f-4e32-8f09-41106c0275a5","Type":"ContainerStarted","Data":"c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2"}
Oct 30 21:47:15.980015 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:15.979988    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:16.109658 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:16.109594    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s" podStartSLOduration=3.040043849 podStartE2EDuration="4.109572143s" totalImagesPullingTime="1.069528294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:16.105751161 +0000 UTC m=+1971.393473706" watchObservedRunningTime="2026-10-30 21:47:16.109572143 +0000 UTC m=+1971.397294688"
Oct 30 21:47:16.487615 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:47:16.487552    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:47:23.051214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:23.051160    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:47:26.453999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:26.453970    5817 patch_prober.go:29] interesting pod/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:47:26.454369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:26.454024    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:47:27.114875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:27.114843    5817 generic.go:356] "Generic (PLEG): container finished" podID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerID="4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23" exitCode=0
Oct 30 21:47:27.115037 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:27.114912    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" event={"ID":"28f25e68-f9e0-469a-85e4-f09007267e9b","Type":"ContainerDied","Data":"4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23"}
Oct 30 21:47:27.115272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:27.115259    5817 scope.go:122] "RemoveContainer" containerID="4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23"
Oct 30 21:47:27.116764 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:27.116736    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:27.116849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:27.116801    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:27.116849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:27.116835    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:47:28.124673 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:28.124635    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" event={"ID":"28f25e68-f9e0-469a-85e4-f09007267e9b","Type":"ContainerStarted","Data":"984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d"}
Oct 30 21:47:28.699431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:28.699392    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:47:28.699784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:28.699761    5817 patch_prober.go:29] interesting pod/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:47:28.699886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:28.699818    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:47:29.708371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:29.707768    5817 patch_prober.go:29] interesting pod/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:47:29.708371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:29.707866    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:47:30.704371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:30.700685    5817 patch_prober.go:29] interesting pod/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:47:30.704371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:30.700765    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:47:31.701718 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:31.701679    5817 patch_prober.go:29] interesting pod/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:47:31.702283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:31.702251    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:47:32.706753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:32.706719    5817 patch_prober.go:29] interesting pod/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:47:32.707309 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:32.707278    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:47:33.707776 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:33.707738    5817 patch_prober.go:29] interesting pod/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 21:47:33.709087 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:33.709055    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 21:47:34.704128 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:34.704097    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:47:34.704639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:34.704616    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:47:34.705190 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:34.705168    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:47:36.524993 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.524965    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"]
Oct 30 21:47:36.528647 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.528627    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.580845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.580801    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-af87dbaf-serving-cert\""
Oct 30 21:47:36.606620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.606593    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"]
Oct 30 21:47:36.622761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.622726    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c72md\" (UniqueName: \"kubernetes.io/projected/4826538a-d05c-46ad-ad06-b693beec405f-kube-api-access-c72md\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.622761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.622758    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4826538a-d05c-46ad-ad06-b693beec405f-openshift-service-ca\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.622928 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.622914    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-metrics-certs\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.622967 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.622941    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-maas-api-tls\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.725063 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.725027    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-metrics-certs\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.725237 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.725075    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-maas-api-tls\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.725340 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.725324    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c72md\" (UniqueName: \"kubernetes.io/projected/4826538a-d05c-46ad-ad06-b693beec405f-kube-api-access-c72md\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.725435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.725365    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4826538a-d05c-46ad-ad06-b693beec405f-openshift-service-ca\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.726395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.726322    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4826538a-d05c-46ad-ad06-b693beec405f-openshift-service-ca\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.728047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.728022    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-maas-api-tls\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.730177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.730157    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-metrics-certs\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.784253 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.784169    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c72md\" (UniqueName: \"kubernetes.io/projected/4826538a-d05c-46ad-ad06-b693beec405f-kube-api-access-c72md\") pod \"maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:36.848207 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:36.848174    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:37.297331 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:37.297301    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"]
Oct 30 21:47:37.297488 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:47:37.297458    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4826538a_d05c_46ad_ad06_b693beec405f.slice/crio-28fac79c4c5e431bb26d22ed3e1555b4314c79c42bad26417cd28ec20e62ac4e WatchSource:0}: Error finding container 28fac79c4c5e431bb26d22ed3e1555b4314c79c42bad26417cd28ec20e62ac4e: Status 404 returned error can't find the container with id 28fac79c4c5e431bb26d22ed3e1555b4314c79c42bad26417cd28ec20e62ac4e
Oct 30 21:47:38.201804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:38.201773    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x" event={"ID":"4826538a-d05c-46ad-ad06-b693beec405f","Type":"ContainerStarted","Data":"0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5"}
Oct 30 21:47:38.202163 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:38.201831    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x" event={"ID":"4826538a-d05c-46ad-ad06-b693beec405f","Type":"ContainerStarted","Data":"28fac79c4c5e431bb26d22ed3e1555b4314c79c42bad26417cd28ec20e62ac4e"}
Oct 30 21:47:38.202163 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:38.201852    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:38.349158 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:38.349105    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x" podStartSLOduration=1.530915508 podStartE2EDuration="2.349091287s" totalImagesPullingTime="818.175779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:38.34722038 +0000 UTC m=+1993.634942924" watchObservedRunningTime="2026-10-30 21:47:38.349091287 +0000 UTC m=+1993.636813831"
Oct 30 21:47:44.214144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:47:44.214112    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:47:46.344678 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:47:46.344631    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:48:10.902492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:10.902460    5817 patch_prober.go:29] interesting pod/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:48:10.902863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:10.902523    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:48:12.127599 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.127560    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"]
Oct 30 21:48:12.240989 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.240941    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"]
Oct 30 21:48:12.610589 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.610556    5817 generic.go:356] "Generic (PLEG): container finished" podID="81ca29db-8554-4880-877a-78576d6e0b78" containerID="1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c" exitCode=0
Oct 30 21:48:12.610830 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.610630    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" event={"ID":"81ca29db-8554-4880-877a-78576d6e0b78","Type":"ContainerDied","Data":"1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c"}
Oct 30 21:48:12.611298 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.611276    5817 scope.go:122] "RemoveContainer" containerID="1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c"
Oct 30 21:48:12.613250 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.613215    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:48:12.613397 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.613319    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:48:12.613478 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:12.613435    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:48:13.624323 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:13.624285    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" event={"ID":"81ca29db-8554-4880-877a-78576d6e0b78","Type":"ContainerStarted","Data":"0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6"}
Oct 30 21:48:13.774242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:13.774210    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:48:13.775283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:13.775254    5817 patch_prober.go:29] interesting pod/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:48:13.775520 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:13.775476    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 21:48:14.771773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:14.771742    5817 patch_prober.go:29] interesting pod/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:48:14.772262 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:14.772226    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 21:48:15.772045 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:15.772007    5817 patch_prober.go:29] interesting pod/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:48:15.772470 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:15.772071    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 21:48:16.343158 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:16.343122    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:48:16.772230 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:16.772201    5817 patch_prober.go:29] interesting pod/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:48:16.772762 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:16.772728    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 21:48:17.128890 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.128775    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4" podUID="260b534a-ef58-4291-a086-f3de6d7b436f" containerName="payload-processing" containerID="cri-o://12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d" gracePeriod=25
Oct 30 21:48:17.252369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.245406    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9" podUID="fe06e60e-3c7a-4026-93aa-9a95f675a56e" containerName="payload-pre-processing" containerID="cri-o://2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4" gracePeriod=25
Oct 30 21:48:17.532244 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.532192    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:48:17.668660 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.668632    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/260b534a-ef58-4291-a086-f3de6d7b436f-kube-api-access-jhtf4\" (UniqueName: \"kubernetes.io/projected/260b534a-ef58-4291-a086-f3de6d7b436f-kube-api-access-jhtf4\") pod \"260b534a-ef58-4291-a086-f3de6d7b436f\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") "
Oct 30 21:48:17.668828 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.668703    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/260b534a-ef58-4291-a086-f3de6d7b436f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/260b534a-ef58-4291-a086-f3de6d7b436f-plugins-config-volume\") pod \"260b534a-ef58-4291-a086-f3de6d7b436f\" (UID: \"260b534a-ef58-4291-a086-f3de6d7b436f\") "
Oct 30 21:48:17.669153 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.669112    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/260b534a-ef58-4291-a086-f3de6d7b436f-plugins-config-volume" pod "260b534a-ef58-4291-a086-f3de6d7b436f" (UID: "260b534a-ef58-4291-a086-f3de6d7b436f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:17.669496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.669473    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/260b534a-ef58-4291-a086-f3de6d7b436f-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:17.670296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.670052    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:48:17.671695 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.671665    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/260b534a-ef58-4291-a086-f3de6d7b436f-kube-api-access-jhtf4" pod "260b534a-ef58-4291-a086-f3de6d7b436f" (UID: "260b534a-ef58-4291-a086-f3de6d7b436f"). InnerVolumeSpecName "kube-api-access-jhtf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:17.675682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.675655    5817 generic.go:356] "Generic (PLEG): container finished" podID="fe06e60e-3c7a-4026-93aa-9a95f675a56e" containerID="2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4" exitCode=0
Oct 30 21:48:17.675784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.675736    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9" event={"ID":"fe06e60e-3c7a-4026-93aa-9a95f675a56e","Type":"ContainerDied","Data":"2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4"}
Oct 30 21:48:17.675860 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.675805    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9" event={"ID":"fe06e60e-3c7a-4026-93aa-9a95f675a56e","Type":"ContainerDied","Data":"37e7afafa4740f15a502d91a74bb204d1926557173e9a28e1a8747da52f3ad40"}
Oct 30 21:48:17.675860 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.675839    5817 scope.go:122] "RemoveContainer" containerID="2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4"
Oct 30 21:48:17.676035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.676008    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"
Oct 30 21:48:17.678032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.678009    5817 generic.go:356] "Generic (PLEG): container finished" podID="260b534a-ef58-4291-a086-f3de6d7b436f" containerID="12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d" exitCode=0
Oct 30 21:48:17.678032 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.678031    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"
Oct 30 21:48:17.678186 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.678081    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4" event={"ID":"260b534a-ef58-4291-a086-f3de6d7b436f","Type":"ContainerDied","Data":"12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d"}
Oct 30 21:48:17.678186 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.678120    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4" event={"ID":"260b534a-ef58-4291-a086-f3de6d7b436f","Type":"ContainerDied","Data":"3469b551f053e2ad762f6909ab81ecd14310a1b6890cba38a0c76f996080954f"}
Oct 30 21:48:17.735681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.735649    5817 scope.go:122] "RemoveContainer" containerID="2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4"
Oct 30 21:48:17.737070 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:17.737035    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4\": container with ID starting with 2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4 not found: ID does not exist" containerID="2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4"
Oct 30 21:48:17.737189 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.737082    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4"} err="failed to get container status \"2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4\": rpc error: code = NotFound desc = could not find container \"2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4\": container with ID starting with 2b7d46c1c2e1db62586eb1e942a0365b80eeb5ca62ae8c1fc5fe88b6e2ff90a4 not found: ID does not exist"
Oct 30 21:48:17.737189 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.737108    5817 scope.go:122] "RemoveContainer" containerID="12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d"
Oct 30 21:48:17.749913 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.749890    5817 scope.go:122] "RemoveContainer" containerID="12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d"
Oct 30 21:48:17.750250 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:17.750223    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d\": container with ID starting with 12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d not found: ID does not exist" containerID="12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d"
Oct 30 21:48:17.750293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.750261    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d"} err="failed to get container status \"12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d\": rpc error: code = NotFound desc = could not find container \"12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d\": container with ID starting with 12644d38b900cbd3086c8ce5e76d56e4cb4285c384ba43e44b18c6aadc67067d not found: ID does not exist"
Oct 30 21:48:17.770135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.770110    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe06e60e-3c7a-4026-93aa-9a95f675a56e-kube-api-access-m78ck\" (UniqueName: \"kubernetes.io/projected/fe06e60e-3c7a-4026-93aa-9a95f675a56e-kube-api-access-m78ck\") pod \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") "
Oct 30 21:48:17.770254 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.770176    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe06e60e-3c7a-4026-93aa-9a95f675a56e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e60e-3c7a-4026-93aa-9a95f675a56e-plugins-config-volume\") pod \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\" (UID: \"fe06e60e-3c7a-4026-93aa-9a95f675a56e\") "
Oct 30 21:48:17.770580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.770554    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe06e60e-3c7a-4026-93aa-9a95f675a56e-plugins-config-volume" pod "fe06e60e-3c7a-4026-93aa-9a95f675a56e" (UID: "fe06e60e-3c7a-4026-93aa-9a95f675a56e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:17.771044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.771025    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe06e60e-3c7a-4026-93aa-9a95f675a56e-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:17.771096 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.771051    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhtf4\" (UniqueName: \"kubernetes.io/projected/260b534a-ef58-4291-a086-f3de6d7b436f-kube-api-access-jhtf4\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:17.771356 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.771326    5817 patch_prober.go:29] interesting pod/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 21:48:17.771415 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.771391    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 21:48:17.772447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.772429    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe06e60e-3c7a-4026-93aa-9a95f675a56e-kube-api-access-m78ck" pod "fe06e60e-3c7a-4026-93aa-9a95f675a56e" (UID: "fe06e60e-3c7a-4026-93aa-9a95f675a56e"). InnerVolumeSpecName "kube-api-access-m78ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:17.872809 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.872776    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m78ck\" (UniqueName: \"kubernetes.io/projected/fe06e60e-3c7a-4026-93aa-9a95f675a56e-kube-api-access-m78ck\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:17.939676 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.939637    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"]
Oct 30 21:48:17.943258 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:17.943231    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e79f61de-6b69866db8-rqth4"]
Oct 30 21:48:18.114490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:18.114442    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"]
Oct 30 21:48:18.118659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:18.118630    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e79f61de-6bcc7fc879-5jmd9"]
Oct 30 21:48:18.778775 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:18.778746    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:48:18.779715 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:18.779294    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:48:18.780068 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:18.780048    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:48:18.845705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:18.845681    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="260b534a-ef58-4291-a086-f3de6d7b436f" path="/var/lib/kubelet/pods/260b534a-ef58-4291-a086-f3de6d7b436f/volumes"
Oct 30 21:48:18.846104 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:18.846090    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe06e60e-3c7a-4026-93aa-9a95f675a56e" path="/var/lib/kubelet/pods/fe06e60e-3c7a-4026-93aa-9a95f675a56e/volumes"
Oct 30 21:48:23.673282 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:23.673221    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"]
Oct 30 21:48:23.793042 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:23.793003    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"]
Oct 30 21:48:28.678372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:28.674419    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp" podUID="355ba04a-1220-46d0-aa21-02389ba9624c" containerName="payload-processing" containerID="cri-o://8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0" gracePeriod=25
Oct 30 21:48:28.801904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:28.801683    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9" podUID="6f4df0b1-b87c-42a7-8d99-db1bd7dde573" containerName="payload-pre-processing" containerID="cri-o://ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040" gracePeriod=25
Oct 30 21:48:29.048489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.048455    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:48:29.146489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.146462    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:48:29.152994 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.152972    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/355ba04a-1220-46d0-aa21-02389ba9624c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355ba04a-1220-46d0-aa21-02389ba9624c-plugins-config-volume\") pod \"355ba04a-1220-46d0-aa21-02389ba9624c\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") "
Oct 30 21:48:29.153139 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.153033    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/355ba04a-1220-46d0-aa21-02389ba9624c-kube-api-access-44l84\" (UniqueName: \"kubernetes.io/projected/355ba04a-1220-46d0-aa21-02389ba9624c-kube-api-access-44l84\") pod \"355ba04a-1220-46d0-aa21-02389ba9624c\" (UID: \"355ba04a-1220-46d0-aa21-02389ba9624c\") "
Oct 30 21:48:29.153422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.153399    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/355ba04a-1220-46d0-aa21-02389ba9624c-plugins-config-volume" pod "355ba04a-1220-46d0-aa21-02389ba9624c" (UID: "355ba04a-1220-46d0-aa21-02389ba9624c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:29.153799 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.153779    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/355ba04a-1220-46d0-aa21-02389ba9624c-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:29.156089 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.156068    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/355ba04a-1220-46d0-aa21-02389ba9624c-kube-api-access-44l84" pod "355ba04a-1220-46d0-aa21-02389ba9624c" (UID: "355ba04a-1220-46d0-aa21-02389ba9624c"). InnerVolumeSpecName "kube-api-access-44l84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:29.254617 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.254582    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-plugins-config-volume\") pod \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") "
Oct 30 21:48:29.254807 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.254718    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-kube-api-access-xx7pr\" (UniqueName: \"kubernetes.io/projected/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-kube-api-access-xx7pr\") pod \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\" (UID: \"6f4df0b1-b87c-42a7-8d99-db1bd7dde573\") "
Oct 30 21:48:29.255034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.255007    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-plugins-config-volume" pod "6f4df0b1-b87c-42a7-8d99-db1bd7dde573" (UID: "6f4df0b1-b87c-42a7-8d99-db1bd7dde573"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:29.255571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.255551    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44l84\" (UniqueName: \"kubernetes.io/projected/355ba04a-1220-46d0-aa21-02389ba9624c-kube-api-access-44l84\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:29.255653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.255573    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:29.257386 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.257365    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-kube-api-access-xx7pr" pod "6f4df0b1-b87c-42a7-8d99-db1bd7dde573" (UID: "6f4df0b1-b87c-42a7-8d99-db1bd7dde573"). InnerVolumeSpecName "kube-api-access-xx7pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:29.357099 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.357058    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx7pr\" (UniqueName: \"kubernetes.io/projected/6f4df0b1-b87c-42a7-8d99-db1bd7dde573-kube-api-access-xx7pr\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:29.789957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.789920    5817 generic.go:356] "Generic (PLEG): container finished" podID="355ba04a-1220-46d0-aa21-02389ba9624c" containerID="8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0" exitCode=0
Oct 30 21:48:29.790493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.789968    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"
Oct 30 21:48:29.790493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.790011    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp" event={"ID":"355ba04a-1220-46d0-aa21-02389ba9624c","Type":"ContainerDied","Data":"8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0"}
Oct 30 21:48:29.790493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.790071    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp" event={"ID":"355ba04a-1220-46d0-aa21-02389ba9624c","Type":"ContainerDied","Data":"076d4846175c08e0de149de910fa25e319a638e6342bba0940338a6f39909258"}
Oct 30 21:48:29.790493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.790104    5817 scope.go:122] "RemoveContainer" containerID="8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0"
Oct 30 21:48:29.791676 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.791656    5817 generic.go:356] "Generic (PLEG): container finished" podID="6f4df0b1-b87c-42a7-8d99-db1bd7dde573" containerID="ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040" exitCode=0
Oct 30 21:48:29.791759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.791700    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"
Oct 30 21:48:29.791759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.791724    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9" event={"ID":"6f4df0b1-b87c-42a7-8d99-db1bd7dde573","Type":"ContainerDied","Data":"ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040"}
Oct 30 21:48:29.791861 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.791765    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9" event={"ID":"6f4df0b1-b87c-42a7-8d99-db1bd7dde573","Type":"ContainerDied","Data":"546e010258be181832c0671f693bf3b1506a3373bda2c0e36b72d020f47b1f37"}
Oct 30 21:48:29.801214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.801189    5817 scope.go:122] "RemoveContainer" containerID="8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0"
Oct 30 21:48:29.801533 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:29.801515    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0\": container with ID starting with 8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0 not found: ID does not exist" containerID="8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0"
Oct 30 21:48:29.801620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.801542    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0"} err="failed to get container status \"8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0\": rpc error: code = NotFound desc = could not find container \"8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0\": container with ID starting with 8b493a3bed2f5944acb3db47562264fb1fd2e9cb7a3f1d8854a048e9a96767f0 not found: ID does not exist"
Oct 30 21:48:29.801620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.801559    5817 scope.go:122] "RemoveContainer" containerID="ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040"
Oct 30 21:48:29.812061 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.812044    5817 scope.go:122] "RemoveContainer" containerID="ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040"
Oct 30 21:48:29.812290 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:29.812271    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040\": container with ID starting with ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040 not found: ID does not exist" containerID="ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040"
Oct 30 21:48:29.812339 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.812296    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040"} err="failed to get container status \"ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040\": rpc error: code = NotFound desc = could not find container \"ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040\": container with ID starting with ad2b7da1633c800b1f58f51b01c388bb1c1e99b9c585d9fd63d65c7abeee6040 not found: ID does not exist"
Oct 30 21:48:29.947067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.947028    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"]
Oct 30 21:48:29.951163 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:29.951132    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a84c1592-64dd7857d4-9jvgp"]
Oct 30 21:48:30.054315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:30.054246    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"]
Oct 30 21:48:30.061520 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:30.061497    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a84c1592-6c97fb985f-wz6r9"]
Oct 30 21:48:30.844565 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:30.844535    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="355ba04a-1220-46d0-aa21-02389ba9624c" path="/var/lib/kubelet/pods/355ba04a-1220-46d0-aa21-02389ba9624c/volumes"
Oct 30 21:48:30.845065 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:30.845048    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f4df0b1-b87c-42a7-8d99-db1bd7dde573" path="/var/lib/kubelet/pods/6f4df0b1-b87c-42a7-8d99-db1bd7dde573/volumes"
Oct 30 21:48:33.940755 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:33.940702    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"]
Oct 30 21:48:34.050057 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:34.050017    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"]
Oct 30 21:48:38.941436 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:38.941395    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz" podUID="15891565-0c25-4a12-acff-68f6aad5dfc3" containerName="payload-processing" containerID="cri-o://e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07" gracePeriod=25
Oct 30 21:48:39.051137 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.051092    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x" podUID="19879e76-bbb7-41b9-a954-71c604c0351d" containerName="payload-pre-processing" containerID="cri-o://7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240" gracePeriod=25
Oct 30 21:48:39.255699 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.255668    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:48:39.353957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.353925    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:48:39.410853 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.410815    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15891565-0c25-4a12-acff-68f6aad5dfc3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15891565-0c25-4a12-acff-68f6aad5dfc3-plugins-config-volume\") pod \"15891565-0c25-4a12-acff-68f6aad5dfc3\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") "
Oct 30 21:48:39.411024 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.410874    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15891565-0c25-4a12-acff-68f6aad5dfc3-kube-api-access-zw7vx\" (UniqueName: \"kubernetes.io/projected/15891565-0c25-4a12-acff-68f6aad5dfc3-kube-api-access-zw7vx\") pod \"15891565-0c25-4a12-acff-68f6aad5dfc3\" (UID: \"15891565-0c25-4a12-acff-68f6aad5dfc3\") "
Oct 30 21:48:39.411222 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.411194    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15891565-0c25-4a12-acff-68f6aad5dfc3-plugins-config-volume" pod "15891565-0c25-4a12-acff-68f6aad5dfc3" (UID: "15891565-0c25-4a12-acff-68f6aad5dfc3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:39.411620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.411599    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15891565-0c25-4a12-acff-68f6aad5dfc3-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:39.413059 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.413034    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15891565-0c25-4a12-acff-68f6aad5dfc3-kube-api-access-zw7vx" pod "15891565-0c25-4a12-acff-68f6aad5dfc3" (UID: "15891565-0c25-4a12-acff-68f6aad5dfc3"). InnerVolumeSpecName "kube-api-access-zw7vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:39.512901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.512831    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19879e76-bbb7-41b9-a954-71c604c0351d-kube-api-access-hbdnz\" (UniqueName: \"kubernetes.io/projected/19879e76-bbb7-41b9-a954-71c604c0351d-kube-api-access-hbdnz\") pod \"19879e76-bbb7-41b9-a954-71c604c0351d\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") "
Oct 30 21:48:39.513058 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.513040    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19879e76-bbb7-41b9-a954-71c604c0351d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19879e76-bbb7-41b9-a954-71c604c0351d-plugins-config-volume\") pod \"19879e76-bbb7-41b9-a954-71c604c0351d\" (UID: \"19879e76-bbb7-41b9-a954-71c604c0351d\") "
Oct 30 21:48:39.513451 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.513424    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19879e76-bbb7-41b9-a954-71c604c0351d-plugins-config-volume" pod "19879e76-bbb7-41b9-a954-71c604c0351d" (UID: "19879e76-bbb7-41b9-a954-71c604c0351d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:39.513747 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.513728    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19879e76-bbb7-41b9-a954-71c604c0351d-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:39.513832 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.513753    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw7vx\" (UniqueName: \"kubernetes.io/projected/15891565-0c25-4a12-acff-68f6aad5dfc3-kube-api-access-zw7vx\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:39.514965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.514945    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19879e76-bbb7-41b9-a954-71c604c0351d-kube-api-access-hbdnz" pod "19879e76-bbb7-41b9-a954-71c604c0351d" (UID: "19879e76-bbb7-41b9-a954-71c604c0351d"). InnerVolumeSpecName "kube-api-access-hbdnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:39.614714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.614689    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbdnz\" (UniqueName: \"kubernetes.io/projected/19879e76-bbb7-41b9-a954-71c604c0351d-kube-api-access-hbdnz\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:39.883543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.883424    5817 generic.go:356] "Generic (PLEG): container finished" podID="15891565-0c25-4a12-acff-68f6aad5dfc3" containerID="e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07" exitCode=0
Oct 30 21:48:39.883543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.883486    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"
Oct 30 21:48:39.883543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.883530    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz" event={"ID":"15891565-0c25-4a12-acff-68f6aad5dfc3","Type":"ContainerDied","Data":"e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07"}
Oct 30 21:48:39.883801 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.883576    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz" event={"ID":"15891565-0c25-4a12-acff-68f6aad5dfc3","Type":"ContainerDied","Data":"f931bdfcfde8e0996656719e307ad05eaa3b280e694b7847c999ded9de59c55c"}
Oct 30 21:48:39.883801 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.883600    5817 scope.go:122] "RemoveContainer" containerID="e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07"
Oct 30 21:48:39.885111 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.885082    5817 generic.go:356] "Generic (PLEG): container finished" podID="19879e76-bbb7-41b9-a954-71c604c0351d" containerID="7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240" exitCode=0
Oct 30 21:48:39.885210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.885143    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"
Oct 30 21:48:39.885210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.885166    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x" event={"ID":"19879e76-bbb7-41b9-a954-71c604c0351d","Type":"ContainerDied","Data":"7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240"}
Oct 30 21:48:39.885316 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.885215    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x" event={"ID":"19879e76-bbb7-41b9-a954-71c604c0351d","Type":"ContainerDied","Data":"3f8315d7271e2fb2d15ea47e08835694af712f68dabb4919ddf57bb4a40ef46d"}
Oct 30 21:48:39.896652 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.896559    5817 scope.go:122] "RemoveContainer" containerID="e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07"
Oct 30 21:48:39.897013 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:39.896987    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07\": container with ID starting with e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07 not found: ID does not exist" containerID="e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07"
Oct 30 21:48:39.897085 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.897023    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07"} err="failed to get container status \"e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07\": rpc error: code = NotFound desc = could not find container \"e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07\": container with ID starting with e78559897fa614141bad523170e9482f603db801a1e77613d2e1fd0dd96eaf07 not found: ID does not exist"
Oct 30 21:48:39.897085 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.897047    5817 scope.go:122] "RemoveContainer" containerID="7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240"
Oct 30 21:48:39.906529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.906510    5817 scope.go:122] "RemoveContainer" containerID="7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240"
Oct 30 21:48:39.906784 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:39.906758    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240\": container with ID starting with 7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240 not found: ID does not exist" containerID="7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240"
Oct 30 21:48:39.906846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:39.906781    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240"} err="failed to get container status \"7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240\": rpc error: code = NotFound desc = could not find container \"7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240\": container with ID starting with 7ba8fece27d1c3a7fd4e91e78d752b073e25d36a7445a020c8c72d81e006e240 not found: ID does not exist"
Oct 30 21:48:40.039787 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:40.039754    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"]
Oct 30 21:48:40.087330 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:40.087302    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-899a6bb1-5c4d949dd4-59fmz"]
Oct 30 21:48:40.147566 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:40.147337    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"]
Oct 30 21:48:40.151277 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:40.151250    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-899a6bb1-575b6c7f4c-jz78x"]
Oct 30 21:48:40.845788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:40.845755    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15891565-0c25-4a12-acff-68f6aad5dfc3" path="/var/lib/kubelet/pods/15891565-0c25-4a12-acff-68f6aad5dfc3/volumes"
Oct 30 21:48:40.846327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:40.846311    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19879e76-bbb7-41b9-a954-71c604c0351d" path="/var/lib/kubelet/pods/19879e76-bbb7-41b9-a954-71c604c0351d/volumes"
Oct 30 21:48:46.338393 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:48:46.338356    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:48:55.714342 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:55.714298    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"]
Oct 30 21:48:55.714882 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:55.714606    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6" podUID="36530fcb-a719-49b1-ba34-c90c63537d68" containerName="maas-api" containerID="cri-o://c1a96c20f88345383bb0376a21df5aeb56705beb7c018d338c646702490327d1" gracePeriod=30
Oct 30 21:48:56.016394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.016312    5817 generic.go:356] "Generic (PLEG): container finished" podID="36530fcb-a719-49b1-ba34-c90c63537d68" containerID="c1a96c20f88345383bb0376a21df5aeb56705beb7c018d338c646702490327d1" exitCode=0
Oct 30 21:48:56.016506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.016482    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6" event={"ID":"36530fcb-a719-49b1-ba34-c90c63537d68","Type":"ContainerDied","Data":"c1a96c20f88345383bb0376a21df5aeb56705beb7c018d338c646702490327d1"}
Oct 30 21:48:56.016573 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.016534    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6" event={"ID":"36530fcb-a719-49b1-ba34-c90c63537d68","Type":"ContainerDied","Data":"964c35c6d2bd1120b23067e68792d0266ecb92bd25fce8da7d8c80fcb4c37a7e"}
Oct 30 21:48:56.016573 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.016559    5817 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="964c35c6d2bd1120b23067e68792d0266ecb92bd25fce8da7d8c80fcb4c37a7e"
Oct 30 21:48:56.040762 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.040737    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:48:56.152659 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.152629    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36530fcb-a719-49b1-ba34-c90c63537d68-kube-api-access-txk9g\" (UniqueName: \"kubernetes.io/projected/36530fcb-a719-49b1-ba34-c90c63537d68-kube-api-access-txk9g\") pod \"36530fcb-a719-49b1-ba34-c90c63537d68\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") "
Oct 30 21:48:56.152854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.152711    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36530fcb-a719-49b1-ba34-c90c63537d68-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36530fcb-a719-49b1-ba34-c90c63537d68-openshift-service-ca\") pod \"36530fcb-a719-49b1-ba34-c90c63537d68\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") "
Oct 30 21:48:56.152924 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.152878    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls\") pod \"36530fcb-a719-49b1-ba34-c90c63537d68\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") "
Oct 30 21:48:56.152998 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.152984    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-metrics-certs\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-metrics-certs\") pod \"36530fcb-a719-49b1-ba34-c90c63537d68\" (UID: \"36530fcb-a719-49b1-ba34-c90c63537d68\") "
Oct 30 21:48:56.153057 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.153012    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36530fcb-a719-49b1-ba34-c90c63537d68-openshift-service-ca" pod "36530fcb-a719-49b1-ba34-c90c63537d68" (UID: "36530fcb-a719-49b1-ba34-c90c63537d68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:56.153464 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.153437    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36530fcb-a719-49b1-ba34-c90c63537d68-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:56.155045 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.155017    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls" pod "36530fcb-a719-49b1-ba34-c90c63537d68" (UID: "36530fcb-a719-49b1-ba34-c90c63537d68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:56.155147 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.155064    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-metrics-certs" pod "36530fcb-a719-49b1-ba34-c90c63537d68" (UID: "36530fcb-a719-49b1-ba34-c90c63537d68"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:48:56.155147 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.155076    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36530fcb-a719-49b1-ba34-c90c63537d68-kube-api-access-txk9g" pod "36530fcb-a719-49b1-ba34-c90c63537d68" (UID: "36530fcb-a719-49b1-ba34-c90c63537d68"). InnerVolumeSpecName "kube-api-access-txk9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:56.254574 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.254498    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:56.254574 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.254527    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txk9g\" (UniqueName: \"kubernetes.io/projected/36530fcb-a719-49b1-ba34-c90c63537d68-kube-api-access-txk9g\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:56.254574 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:56.254539    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36530fcb-a719-49b1-ba34-c90c63537d68-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:48:57.024641 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:57.024608    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"
Oct 30 21:48:57.181649 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:57.181558    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"]
Oct 30 21:48:57.228654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:57.228628    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e79f61de-6c58bbcd4-pf4t6"]
Oct 30 21:48:58.034139 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:58.034103    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"]
Oct 30 21:48:58.034577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:58.034312    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy" containerID="cri-o://984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d" gracePeriod=30
Oct 30 21:48:58.846965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:58.846931    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36530fcb-a719-49b1-ba34-c90c63537d68" path="/var/lib/kubelet/pods/36530fcb-a719-49b1-ba34-c90c63537d68/volumes"
Oct 30 21:48:59.673511 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.673465    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"]
Oct 30 21:48:59.674666 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674646    5817 cpu_manager.go:414] "containerMap: removing container" podUID="36530fcb-a719-49b1-ba34-c90c63537d68" containerName="maas-api"
Oct 30 21:48:59.674741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674669    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36530fcb-a719-49b1-ba34-c90c63537d68" containerName="maas-api"
Oct 30 21:48:59.674741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674686    5817 cpu_manager.go:414] "containerMap: removing container" podUID="fe06e60e-3c7a-4026-93aa-9a95f675a56e" containerName="payload-pre-processing"
Oct 30 21:48:59.674741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674694    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe06e60e-3c7a-4026-93aa-9a95f675a56e" containerName="payload-pre-processing"
Oct 30 21:48:59.674741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674716    5817 cpu_manager.go:414] "containerMap: removing container" podUID="355ba04a-1220-46d0-aa21-02389ba9624c" containerName="payload-processing"
Oct 30 21:48:59.674741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674724    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="355ba04a-1220-46d0-aa21-02389ba9624c" containerName="payload-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674759    5817 cpu_manager.go:414] "containerMap: removing container" podUID="6f4df0b1-b87c-42a7-8d99-db1bd7dde573" containerName="payload-pre-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674767    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f4df0b1-b87c-42a7-8d99-db1bd7dde573" containerName="payload-pre-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674780    5817 cpu_manager.go:414] "containerMap: removing container" podUID="19879e76-bbb7-41b9-a954-71c604c0351d" containerName="payload-pre-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674787    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19879e76-bbb7-41b9-a954-71c604c0351d" containerName="payload-pre-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674837    5817 cpu_manager.go:414] "containerMap: removing container" podUID="260b534a-ef58-4291-a086-f3de6d7b436f" containerName="payload-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674845    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="260b534a-ef58-4291-a086-f3de6d7b436f" containerName="payload-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674858    5817 cpu_manager.go:414] "containerMap: removing container" podUID="15891565-0c25-4a12-acff-68f6aad5dfc3" containerName="payload-processing"
Oct 30 21:48:59.674894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.674865    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15891565-0c25-4a12-acff-68f6aad5dfc3" containerName="payload-processing"
Oct 30 21:48:59.675134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.675039    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="260b534a-ef58-4291-a086-f3de6d7b436f" containerName="payload-processing"
Oct 30 21:48:59.675134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.675052    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="15891565-0c25-4a12-acff-68f6aad5dfc3" containerName="payload-processing"
Oct 30 21:48:59.675134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.675065    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="36530fcb-a719-49b1-ba34-c90c63537d68" containerName="maas-api"
Oct 30 21:48:59.675134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.675077    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe06e60e-3c7a-4026-93aa-9a95f675a56e" containerName="payload-pre-processing"
Oct 30 21:48:59.675134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.675089    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="355ba04a-1220-46d0-aa21-02389ba9624c" containerName="payload-processing"
Oct 30 21:48:59.675134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.675102    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f4df0b1-b87c-42a7-8d99-db1bd7dde573" containerName="payload-pre-processing"
Oct 30 21:48:59.675134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.675114    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="19879e76-bbb7-41b9-a954-71c604c0351d" containerName="payload-pre-processing"
Oct 30 21:48:59.680616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.680588    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.733196 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.733163    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fbdbd87a-gw-openshift-default-dockercfg-m6rxk\""
Oct 30 21:48:59.782460 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.782427    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"]
Oct 30 21:48:59.813298 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.813245    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-istio-token\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.813599 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.813577    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-data\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.813777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.813741    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-credential-socket\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.813934 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.813794    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtwsp\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-kube-api-access-xtwsp\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.814003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.813945    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-socket\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.814064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.814036    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bace0d0d-712e-4138-88da-98c90e39413a-istio-podinfo\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.814121 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.814090    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-certs\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.814203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.814184    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-envoy\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.814301 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.814284    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bace0d0d-712e-4138-88da-98c90e39413a-istiod-ca-cert\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916381 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916305    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bace0d0d-712e-4138-88da-98c90e39413a-istiod-ca-cert\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916454    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-istio-token\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916507    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-data\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916711 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916591    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-credential-socket\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916711 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916625    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtwsp\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-kube-api-access-xtwsp\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916731    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-socket\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916872 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916827    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bace0d0d-712e-4138-88da-98c90e39413a-istio-podinfo\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.916935 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916885    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-certs\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.917003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916981    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-data\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.917060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.917010    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-credential-socket\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.917115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.917102    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bace0d0d-712e-4138-88da-98c90e39413a-istiod-ca-cert\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.917168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.916991    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-envoy\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.917396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.917293    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-socket\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.917744 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.917702    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-certs\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.919526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.919496    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-envoy\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.919963 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.919939    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bace0d0d-712e-4138-88da-98c90e39413a-istio-podinfo\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.977416 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.977376    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-istio-token\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.978825 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.978791    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtwsp\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-kube-api-access-xtwsp\") pod \"e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") " pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:48:59.998417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:48:59.995864    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:49:00.437769 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:49:00.437737    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbace0d0d_712e_4138_88da_98c90e39413a.slice/crio-dcd76149b46ba7f1d10eb9c3e86a34a9f9e98a38b2ce929fc23dde0d227ef587 WatchSource:0}: Error finding container dcd76149b46ba7f1d10eb9c3e86a34a9f9e98a38b2ce929fc23dde0d227ef587: Status 404 returned error can't find the container with id dcd76149b46ba7f1d10eb9c3e86a34a9f9e98a38b2ce929fc23dde0d227ef587
Oct 30 21:49:00.437923 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:00.437871    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"]
Oct 30 21:49:00.440426 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:00.440390    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:00.440504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:00.440465    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:00.440504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:00.440495    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:01.065016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:01.064977    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" event={"ID":"bace0d0d-712e-4138-88da-98c90e39413a","Type":"ContainerStarted","Data":"4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02"}
Oct 30 21:49:01.065546 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:01.065050    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" event={"ID":"bace0d0d-712e-4138-88da-98c90e39413a","Type":"ContainerStarted","Data":"dcd76149b46ba7f1d10eb9c3e86a34a9f9e98a38b2ce929fc23dde0d227ef587"}
Oct 30 21:49:01.129334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:01.129288    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"]
Oct 30 21:49:01.184037 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:01.183976    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podStartSLOduration=2.183961329 podStartE2EDuration="2.183961329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:01.182207511 +0000 UTC m=+2076.469930057" watchObservedRunningTime="2026-10-30 21:49:01.183961329 +0000 UTC m=+2076.471683878"
Oct 30 21:49:01.243059 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:01.243021    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"]
Oct 30 21:49:01.997398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:01.997328    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:49:02.004006 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.003980    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:49:02.071982 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.071951    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:49:02.072726 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.072710    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:49:02.249856 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.249740    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"]
Oct 30 21:49:02.250003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.249972    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9" podUID="57d49345-36ce-4dae-b373-e78926d7ba49" containerName="maas-api" containerID="cri-o://172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04" gracePeriod=30
Oct 30 21:49:02.555046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.555015    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:49:02.606750 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.606710    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-metrics-certs\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-metrics-certs\") pod \"57d49345-36ce-4dae-b373-e78926d7ba49\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") "
Oct 30 21:49:02.606949 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.606768    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57d49345-36ce-4dae-b373-e78926d7ba49-kube-api-access-8p2gs\" (UniqueName: \"kubernetes.io/projected/57d49345-36ce-4dae-b373-e78926d7ba49-kube-api-access-8p2gs\") pod \"57d49345-36ce-4dae-b373-e78926d7ba49\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") "
Oct 30 21:49:02.606949 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.606875    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57d49345-36ce-4dae-b373-e78926d7ba49-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d49345-36ce-4dae-b373-e78926d7ba49-openshift-service-ca\") pod \"57d49345-36ce-4dae-b373-e78926d7ba49\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") "
Oct 30 21:49:02.606949 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.606930    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls\") pod \"57d49345-36ce-4dae-b373-e78926d7ba49\" (UID: \"57d49345-36ce-4dae-b373-e78926d7ba49\") "
Oct 30 21:49:02.607278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.607248    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57d49345-36ce-4dae-b373-e78926d7ba49-openshift-service-ca" pod "57d49345-36ce-4dae-b373-e78926d7ba49" (UID: "57d49345-36ce-4dae-b373-e78926d7ba49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:02.607688 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.607667    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57d49345-36ce-4dae-b373-e78926d7ba49-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:02.609133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.609109    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-metrics-certs" pod "57d49345-36ce-4dae-b373-e78926d7ba49" (UID: "57d49345-36ce-4dae-b373-e78926d7ba49"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:02.609238 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.609190    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls" pod "57d49345-36ce-4dae-b373-e78926d7ba49" (UID: "57d49345-36ce-4dae-b373-e78926d7ba49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:02.609238 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.609193    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57d49345-36ce-4dae-b373-e78926d7ba49-kube-api-access-8p2gs" pod "57d49345-36ce-4dae-b373-e78926d7ba49" (UID: "57d49345-36ce-4dae-b373-e78926d7ba49"). InnerVolumeSpecName "kube-api-access-8p2gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:02.724002 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.723149    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:02.724625 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.724599    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p2gs\" (UniqueName: \"kubernetes.io/projected/57d49345-36ce-4dae-b373-e78926d7ba49-kube-api-access-8p2gs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:02.724805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:02.724790    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57d49345-36ce-4dae-b373-e78926d7ba49-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.097084 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.097047    5817 generic.go:356] "Generic (PLEG): container finished" podID="57d49345-36ce-4dae-b373-e78926d7ba49" containerID="172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04" exitCode=0
Oct 30 21:49:03.097581 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.097103    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"
Oct 30 21:49:03.097581 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.097130    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9" event={"ID":"57d49345-36ce-4dae-b373-e78926d7ba49","Type":"ContainerDied","Data":"172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04"}
Oct 30 21:49:03.097581 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.097217    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9" event={"ID":"57d49345-36ce-4dae-b373-e78926d7ba49","Type":"ContainerDied","Data":"c99148c129c72056acaaa0139f0af6a02c5d4ffde89fde9297d620edb7c3222b"}
Oct 30 21:49:03.097581 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.097249    5817 scope.go:122] "RemoveContainer" containerID="172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04"
Oct 30 21:49:03.149665 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.149633    5817 scope.go:122] "RemoveContainer" containerID="172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04"
Oct 30 21:49:03.150010 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:03.149987    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04\": container with ID starting with 172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04 not found: ID does not exist" containerID="172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04"
Oct 30 21:49:03.150120 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.150018    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04"} err="failed to get container status \"172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04\": rpc error: code = NotFound desc = could not find container \"172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04\": container with ID starting with 172140982c80f69d4d28cbb2100b647c8c51cef8b267f41d0b52b01c6b78ea04 not found: ID does not exist"
Oct 30 21:49:03.259290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.259257    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"]
Oct 30 21:49:03.264922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.264872    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a84c1592-784684ffc6-mmsd9"]
Oct 30 21:49:03.472214 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.472169    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:49:03.554636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554604    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-envoy\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.554826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554699    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-data\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.554826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554730    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-kube-api-access-jx75j\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-kube-api-access-jx75j\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.554826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554770    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-socket\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.554995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554858    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-certs\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.554995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554894    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-credential-socket\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.554995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554954    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-data" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:03.554995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.554962    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28f25e68-f9e0-469a-85e4-f09007267e9b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f25e68-f9e0-469a-85e4-f09007267e9b-istiod-ca-cert\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.555205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.555149    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-socket" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:03.555261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.555163    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-certs" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:03.555261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.555203    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-credential-socket" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:03.555261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.555244    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-podinfo\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.555435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.555378    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-token\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-token\") pod \"28f25e68-f9e0-469a-85e4-f09007267e9b\" (UID: \"28f25e68-f9e0-469a-85e4-f09007267e9b\") "
Oct 30 21:49:03.555435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.555386    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f25e68-f9e0-469a-85e4-f09007267e9b-istiod-ca-cert" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:03.556255 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.556225    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28f25e68-f9e0-469a-85e4-f09007267e9b-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.556458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.556258    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.556458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.556273    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.556458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.556287    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.556458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.556301    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.557110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.557085    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-envoy" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:03.557309 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.557280    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-kube-api-access-jx75j" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "kube-api-access-jx75j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:03.557454 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.557335    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-podinfo" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:03.557987 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.557967    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-token" pod "28f25e68-f9e0-469a-85e4-f09007267e9b" (UID: "28f25e68-f9e0-469a-85e4-f09007267e9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:03.657918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.657889    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.657918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.657917    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.657918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.657926    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28f25e68-f9e0-469a-85e4-f09007267e9b-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:03.658131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:03.657935    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx75j\" (UniqueName: \"kubernetes.io/projected/28f25e68-f9e0-469a-85e4-f09007267e9b-kube-api-access-jx75j\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:04.107337 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.107300    5817 generic.go:356] "Generic (PLEG): container finished" podID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerID="984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d" exitCode=0
Oct 30 21:49:04.107845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.107390    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"
Oct 30 21:49:04.107845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.107385    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" event={"ID":"28f25e68-f9e0-469a-85e4-f09007267e9b","Type":"ContainerDied","Data":"984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d"}
Oct 30 21:49:04.107845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.107514    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf" event={"ID":"28f25e68-f9e0-469a-85e4-f09007267e9b","Type":"ContainerDied","Data":"589a01e2573d0682425c662091404ef0320f6f0187ade3c73fbb1c4b058922f8"}
Oct 30 21:49:04.107845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.107536    5817 scope.go:122] "RemoveContainer" containerID="984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d"
Oct 30 21:49:04.119125 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.119105    5817 scope.go:122] "RemoveContainer" containerID="4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23"
Oct 30 21:49:04.130116 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.130095    5817 scope.go:122] "RemoveContainer" containerID="984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d"
Oct 30 21:49:04.130430 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:04.130406    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d\": container with ID starting with 984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d not found: ID does not exist" containerID="984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d"
Oct 30 21:49:04.130553 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.130440    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d"} err="failed to get container status \"984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d\": rpc error: code = NotFound desc = could not find container \"984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d\": container with ID starting with 984a74dadee9d4da74cf7f226b68fbdab3b5bb28df9ee67b4d91423bf5fdc68d not found: ID does not exist"
Oct 30 21:49:04.130553 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.130464    5817 scope.go:122] "RemoveContainer" containerID="4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23"
Oct 30 21:49:04.130747 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:04.130730    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23\": container with ID starting with 4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23 not found: ID does not exist" containerID="4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23"
Oct 30 21:49:04.130803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.130759    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23"} err="failed to get container status \"4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23\": rpc error: code = NotFound desc = could not find container \"4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23\": container with ID starting with 4d894c20a74f9c04f991c6ab0fac9704f85bf0139e99c63b0fa49b45b2bc8e23 not found: ID does not exist"
Oct 30 21:49:04.277611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.277581    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"]
Oct 30 21:49:04.323756 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.323721    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e79f61de-openshift-default-85cf8fbb7f-zzmdf"]
Oct 30 21:49:04.845582 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.845546    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" path="/var/lib/kubelet/pods/28f25e68-f9e0-469a-85e4-f09007267e9b/volumes"
Oct 30 21:49:04.846158 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:04.846139    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57d49345-36ce-4dae-b373-e78926d7ba49" path="/var/lib/kubelet/pods/57d49345-36ce-4dae-b373-e78926d7ba49/volumes"
Oct 30 21:49:06.130109 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.130071    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m" podUID="7db130db-5113-40f1-a7de-4f4c2bf45b56" containerName="payload-processing" containerID="cri-o://ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36" gracePeriod=25
Oct 30 21:49:06.243333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.243295    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm" podUID="6c75a704-fcbe-48f8-811c-0a64c8193acf" containerName="payload-pre-processing" containerID="cri-o://4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3" gracePeriod=25
Oct 30 21:49:06.543679 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.541563    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:49:06.623392 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.622080    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7db130db-5113-40f1-a7de-4f4c2bf45b56-kube-api-access-bb8xb\" (UniqueName: \"kubernetes.io/projected/7db130db-5113-40f1-a7de-4f4c2bf45b56-kube-api-access-bb8xb\") pod \"7db130db-5113-40f1-a7de-4f4c2bf45b56\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") "
Oct 30 21:49:06.623392 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.622375    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7db130db-5113-40f1-a7de-4f4c2bf45b56-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db130db-5113-40f1-a7de-4f4c2bf45b56-plugins-config-volume\") pod \"7db130db-5113-40f1-a7de-4f4c2bf45b56\" (UID: \"7db130db-5113-40f1-a7de-4f4c2bf45b56\") "
Oct 30 21:49:06.651370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.625853    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7db130db-5113-40f1-a7de-4f4c2bf45b56-plugins-config-volume" pod "7db130db-5113-40f1-a7de-4f4c2bf45b56" (UID: "7db130db-5113-40f1-a7de-4f4c2bf45b56"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:06.675193 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.672562    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"]
Oct 30 21:49:06.675752 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.673677    5817 cpu_manager.go:414] "containerMap: removing container" podUID="7db130db-5113-40f1-a7de-4f4c2bf45b56" containerName="payload-processing"
Oct 30 21:49:06.675752 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.675655    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7db130db-5113-40f1-a7de-4f4c2bf45b56" containerName="payload-processing"
Oct 30 21:49:06.675752 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.675681    5817 cpu_manager.go:414] "containerMap: removing container" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy"
Oct 30 21:49:06.675752 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.675700    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy"
Oct 30 21:49:06.676044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.675780    5817 cpu_manager.go:414] "containerMap: removing container" podUID="57d49345-36ce-4dae-b373-e78926d7ba49" containerName="maas-api"
Oct 30 21:49:06.676044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.675790    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57d49345-36ce-4dae-b373-e78926d7ba49" containerName="maas-api"
Oct 30 21:49:06.676202 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.676179    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy"
Oct 30 21:49:06.676283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.676207    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="57d49345-36ce-4dae-b373-e78926d7ba49" containerName="maas-api"
Oct 30 21:49:06.676283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.676231    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="7db130db-5113-40f1-a7de-4f4c2bf45b56" containerName="payload-processing"
Oct 30 21:49:06.676283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.676242    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy"
Oct 30 21:49:06.678187 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.674472    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7db130db-5113-40f1-a7de-4f4c2bf45b56-kube-api-access-bb8xb" pod "7db130db-5113-40f1-a7de-4f4c2bf45b56" (UID: "7db130db-5113-40f1-a7de-4f4c2bf45b56"). InnerVolumeSpecName "kube-api-access-bb8xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:06.682728 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.682707    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:06.691997 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.691968    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:49:06.704736 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.704709    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"]
Oct 30 21:49:06.724168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.724143    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c75a704-fcbe-48f8-811c-0a64c8193acf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c75a704-fcbe-48f8-811c-0a64c8193acf-plugins-config-volume\") pod \"6c75a704-fcbe-48f8-811c-0a64c8193acf\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") "
Oct 30 21:49:06.724293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.724264    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c75a704-fcbe-48f8-811c-0a64c8193acf-kube-api-access-p8q2b\" (UniqueName: \"kubernetes.io/projected/6c75a704-fcbe-48f8-811c-0a64c8193acf-kube-api-access-p8q2b\") pod \"6c75a704-fcbe-48f8-811c-0a64c8193acf\" (UID: \"6c75a704-fcbe-48f8-811c-0a64c8193acf\") "
Oct 30 21:49:06.724790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.724764    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:06.724915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.724899    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7hsg\" (UniqueName: \"kubernetes.io/projected/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-kube-api-access-t7hsg\") pod \"payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:06.725136 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.725120    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db130db-5113-40f1-a7de-4f4c2bf45b56-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:06.725223 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.725141    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb8xb\" (UniqueName: \"kubernetes.io/projected/7db130db-5113-40f1-a7de-4f4c2bf45b56-kube-api-access-bb8xb\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:06.725951 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.725928    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c75a704-fcbe-48f8-811c-0a64c8193acf-plugins-config-volume" pod "6c75a704-fcbe-48f8-811c-0a64c8193acf" (UID: "6c75a704-fcbe-48f8-811c-0a64c8193acf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:06.731993 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.731970    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-fbdbd87a\""
Oct 30 21:49:06.732292 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.732256    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-fbdbd87a-dockercfg-wkbpw\""
Oct 30 21:49:06.756371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.753326    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c75a704-fcbe-48f8-811c-0a64c8193acf-kube-api-access-p8q2b" pod "6c75a704-fcbe-48f8-811c-0a64c8193acf" (UID: "6c75a704-fcbe-48f8-811c-0a64c8193acf"). InnerVolumeSpecName "kube-api-access-p8q2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:06.826490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.820304    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"]
Oct 30 21:49:06.826490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.821548    5817 cpu_manager.go:414] "containerMap: removing container" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy"
Oct 30 21:49:06.826490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.821571    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28f25e68-f9e0-469a-85e4-f09007267e9b" containerName="istio-proxy"
Oct 30 21:49:06.826490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.821654    5817 cpu_manager.go:414] "containerMap: removing container" podUID="6c75a704-fcbe-48f8-811c-0a64c8193acf" containerName="payload-pre-processing"
Oct 30 21:49:06.826490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.821667    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c75a704-fcbe-48f8-811c-0a64c8193acf" containerName="payload-pre-processing"
Oct 30 21:49:06.826490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.821910    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c75a704-fcbe-48f8-811c-0a64c8193acf" containerName="payload-pre-processing"
Oct 30 21:49:06.851373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.843582    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7hsg\" (UniqueName: \"kubernetes.io/projected/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-kube-api-access-t7hsg\") pod \"payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:06.851373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.844036    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:06.851373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.844209    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c75a704-fcbe-48f8-811c-0a64c8193acf-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:06.851373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.844239    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8q2b\" (UniqueName: \"kubernetes.io/projected/6c75a704-fcbe-48f8-811c-0a64c8193acf-kube-api-access-p8q2b\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:06.851373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.845156    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:06.866088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.862454    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:06.947994 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.947953    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ab6413-5433-495d-b60c-bfb93a7b58df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") " pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:06.948155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.948134    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99r6q\" (UniqueName: \"kubernetes.io/projected/a4ab6413-5433-495d-b60c-bfb93a7b58df-kube-api-access-99r6q\") pod \"payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") " pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:06.954568 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.954534    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7hsg\" (UniqueName: \"kubernetes.io/projected/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-kube-api-access-t7hsg\") pod \"payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:06.966908 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:06.965936    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"]
Oct 30 21:49:07.018933 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.018908    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:07.049326 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.049293    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99r6q\" (UniqueName: \"kubernetes.io/projected/a4ab6413-5433-495d-b60c-bfb93a7b58df-kube-api-access-99r6q\") pod \"payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") " pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:07.049653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.049635    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ab6413-5433-495d-b60c-bfb93a7b58df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") " pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:07.050386 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.050324    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ab6413-5433-495d-b60c-bfb93a7b58df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") " pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:07.107576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.107544    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99r6q\" (UniqueName: \"kubernetes.io/projected/a4ab6413-5433-495d-b60c-bfb93a7b58df-kube-api-access-99r6q\") pod \"payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") " pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:07.135148 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.135116    5817 generic.go:356] "Generic (PLEG): container finished" podID="7db130db-5113-40f1-a7de-4f4c2bf45b56" containerID="ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36" exitCode=0
Oct 30 21:49:07.135663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.135195    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"
Oct 30 21:49:07.135663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.135201    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m" event={"ID":"7db130db-5113-40f1-a7de-4f4c2bf45b56","Type":"ContainerDied","Data":"ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36"}
Oct 30 21:49:07.135663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.135268    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m" event={"ID":"7db130db-5113-40f1-a7de-4f4c2bf45b56","Type":"ContainerDied","Data":"2ff4a24468bfe1454f9e569c71c8ea357b8d05253f51c961afa0b7c5e15533ed"}
Oct 30 21:49:07.135663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.135304    5817 scope.go:122] "RemoveContainer" containerID="ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36"
Oct 30 21:49:07.137452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.137425    5817 generic.go:356] "Generic (PLEG): container finished" podID="6c75a704-fcbe-48f8-811c-0a64c8193acf" containerID="4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3" exitCode=0
Oct 30 21:49:07.137729 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.137695    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm" event={"ID":"6c75a704-fcbe-48f8-811c-0a64c8193acf","Type":"ContainerDied","Data":"4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3"}
Oct 30 21:49:07.137868 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.137707    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"
Oct 30 21:49:07.138261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.138213    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm" event={"ID":"6c75a704-fcbe-48f8-811c-0a64c8193acf","Type":"ContainerDied","Data":"fb83ee2cf1b9a9934cbd4a5b556c48e349d32456bdb308b0fddaf821d8f0f5ee"}
Oct 30 21:49:07.175309 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.175286    5817 scope.go:122] "RemoveContainer" containerID="ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36"
Oct 30 21:49:07.175663 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:07.175633    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36\": container with ID starting with ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36 not found: ID does not exist" containerID="ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36"
Oct 30 21:49:07.175754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.175675    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36"} err="failed to get container status \"ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36\": rpc error: code = NotFound desc = could not find container \"ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36\": container with ID starting with ade6513e0041954bfb933bae711727289922123d185135ed454ffbd326aeec36 not found: ID does not exist"
Oct 30 21:49:07.175754 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.175699    5817 scope.go:122] "RemoveContainer" containerID="4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3"
Oct 30 21:49:07.185165 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.185147    5817 scope.go:122] "RemoveContainer" containerID="4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3"
Oct 30 21:49:07.185477 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:07.185451    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3\": container with ID starting with 4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3 not found: ID does not exist" containerID="4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3"
Oct 30 21:49:07.185530 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.185484    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3"} err="failed to get container status \"4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3\": rpc error: code = NotFound desc = could not find container \"4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3\": container with ID starting with 4a5f2c61da575d7ec7e0dc846e377015f15c4c68cb213f874f4f77bdb52d55d3 not found: ID does not exist"
Oct 30 21:49:07.265922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.265900    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:07.294655 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.294629    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"]
Oct 30 21:49:07.299088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.299057    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af87dbaf-867f7d5d76-dj28m"]
Oct 30 21:49:07.410453 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.407779    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"]
Oct 30 21:49:07.410453 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.407852    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"]
Oct 30 21:49:07.410453 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:49:07.409557    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ee7ba8_9f02_4ac1_af48_02e29f24f82d.slice/crio-a9bb4b3c797df530c188ca146eb09f1f9c989c1b7ec2edcaac7508821a89373c WatchSource:0}: Error finding container a9bb4b3c797df530c188ca146eb09f1f9c989c1b7ec2edcaac7508821a89373c: Status 404 returned error can't find the container with id a9bb4b3c797df530c188ca146eb09f1f9c989c1b7ec2edcaac7508821a89373c
Oct 30 21:49:07.424596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.422140    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af87dbaf-95cd4d4dc-zphmm"]
Oct 30 21:49:07.699010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:07.698055    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"]
Oct 30 21:49:07.706579 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:49:07.706540    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ab6413_5433_495d_b60c_bfb93a7b58df.slice/crio-9476eaee9fb687bf28b4e4c5a5d998529c589a9825dfe83c2d56ecaf40519878 WatchSource:0}: Error finding container 9476eaee9fb687bf28b4e4c5a5d998529c589a9825dfe83c2d56ecaf40519878: Status 404 returned error can't find the container with id 9476eaee9fb687bf28b4e4c5a5d998529c589a9825dfe83c2d56ecaf40519878
Oct 30 21:49:08.146363 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.146310    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2" event={"ID":"80ee7ba8-9f02-4ac1-af48-02e29f24f82d","Type":"ContainerStarted","Data":"ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8"}
Oct 30 21:49:08.146796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.146387    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2" event={"ID":"80ee7ba8-9f02-4ac1-af48-02e29f24f82d","Type":"ContainerStarted","Data":"a9bb4b3c797df530c188ca146eb09f1f9c989c1b7ec2edcaac7508821a89373c"}
Oct 30 21:49:08.146796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.146419    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:08.148255 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.148225    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7" event={"ID":"a4ab6413-5433-495d-b60c-bfb93a7b58df","Type":"ContainerStarted","Data":"46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0"}
Oct 30 21:49:08.148423 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.148272    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7" event={"ID":"a4ab6413-5433-495d-b60c-bfb93a7b58df","Type":"ContainerStarted","Data":"9476eaee9fb687bf28b4e4c5a5d998529c589a9825dfe83c2d56ecaf40519878"}
Oct 30 21:49:08.148423 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.148300    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:08.261503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.261436    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2" podStartSLOduration=2.261417822 podStartE2EDuration="2.261417822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:08.257893062 +0000 UTC m=+2083.545615609" watchObservedRunningTime="2026-10-30 21:49:08.261417822 +0000 UTC m=+2083.549140367"
Oct 30 21:49:08.373173 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.373121    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7" podStartSLOduration=2.373106037 podStartE2EDuration="2.373106037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:08.37223337 +0000 UTC m=+2083.659955918" watchObservedRunningTime="2026-10-30 21:49:08.373106037 +0000 UTC m=+2083.660828583"
Oct 30 21:49:08.845722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.845685    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c75a704-fcbe-48f8-811c-0a64c8193acf" path="/var/lib/kubelet/pods/6c75a704-fcbe-48f8-811c-0a64c8193acf/volumes"
Oct 30 21:49:08.846170 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:08.846156    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7db130db-5113-40f1-a7de-4f4c2bf45b56" path="/var/lib/kubelet/pods/7db130db-5113-40f1-a7de-4f4c2bf45b56/volumes"
Oct 30 21:49:13.604679 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.604640    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"]
Oct 30 21:49:13.605240 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.604930    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s" podUID="45459602-ee7f-4e32-8f09-41106c0275a5" containerName="maas-api" containerID="cri-o://c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2" gracePeriod=30
Oct 30 21:49:13.921362 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.921313    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:49:13.974246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.974205    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls\") pod \"45459602-ee7f-4e32-8f09-41106c0275a5\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") "
Oct 30 21:49:13.974450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.974276    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45459602-ee7f-4e32-8f09-41106c0275a5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45459602-ee7f-4e32-8f09-41106c0275a5-openshift-service-ca\") pod \"45459602-ee7f-4e32-8f09-41106c0275a5\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") "
Oct 30 21:49:13.974450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.974340    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-metrics-certs\") pod \"45459602-ee7f-4e32-8f09-41106c0275a5\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") "
Oct 30 21:49:13.974450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.974424    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45459602-ee7f-4e32-8f09-41106c0275a5-kube-api-access-szx9h\" (UniqueName: \"kubernetes.io/projected/45459602-ee7f-4e32-8f09-41106c0275a5-kube-api-access-szx9h\") pod \"45459602-ee7f-4e32-8f09-41106c0275a5\" (UID: \"45459602-ee7f-4e32-8f09-41106c0275a5\") "
Oct 30 21:49:13.975538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.975031    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45459602-ee7f-4e32-8f09-41106c0275a5-openshift-service-ca" pod "45459602-ee7f-4e32-8f09-41106c0275a5" (UID: "45459602-ee7f-4e32-8f09-41106c0275a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:13.975538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.975500    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45459602-ee7f-4e32-8f09-41106c0275a5-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:13.977487 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.977461    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45459602-ee7f-4e32-8f09-41106c0275a5-kube-api-access-szx9h" pod "45459602-ee7f-4e32-8f09-41106c0275a5" (UID: "45459602-ee7f-4e32-8f09-41106c0275a5"). InnerVolumeSpecName "kube-api-access-szx9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:13.981854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.981826    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls" pod "45459602-ee7f-4e32-8f09-41106c0275a5" (UID: "45459602-ee7f-4e32-8f09-41106c0275a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:13.982157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:13.982119    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-metrics-certs" pod "45459602-ee7f-4e32-8f09-41106c0275a5" (UID: "45459602-ee7f-4e32-8f09-41106c0275a5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:14.077052 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.077014    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:14.077052 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.077048    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45459602-ee7f-4e32-8f09-41106c0275a5-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:14.077052 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.077058    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szx9h\" (UniqueName: \"kubernetes.io/projected/45459602-ee7f-4e32-8f09-41106c0275a5-kube-api-access-szx9h\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:14.198068 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.197978    5817 generic.go:356] "Generic (PLEG): container finished" podID="45459602-ee7f-4e32-8f09-41106c0275a5" containerID="c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2" exitCode=0
Oct 30 21:49:14.198068 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.198028    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"
Oct 30 21:49:14.198068 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.198057    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s" event={"ID":"45459602-ee7f-4e32-8f09-41106c0275a5","Type":"ContainerDied","Data":"c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2"}
Oct 30 21:49:14.198308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.198126    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s" event={"ID":"45459602-ee7f-4e32-8f09-41106c0275a5","Type":"ContainerDied","Data":"7826ddcf468f0bfcef18e8b3626a05632049bc3a70b07a671fac67e4a3b59833"}
Oct 30 21:49:14.198308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.198156    5817 scope.go:122] "RemoveContainer" containerID="c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2"
Oct 30 21:49:14.212691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.212669    5817 scope.go:122] "RemoveContainer" containerID="c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2"
Oct 30 21:49:14.213092 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:14.213053    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2\": container with ID starting with c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2 not found: ID does not exist" containerID="c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2"
Oct 30 21:49:14.213182 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.213097    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2"} err="failed to get container status \"c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2\": rpc error: code = NotFound desc = could not find container \"c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2\": container with ID starting with c17a784ab569f4d87e936ae4a9287ced484410c637834bcb73952875e313acf2 not found: ID does not exist"
Oct 30 21:49:14.380583 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.380546    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"]
Oct 30 21:49:14.399616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.399544    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-899a6bb1-5886594fc6-fhg6s"]
Oct 30 21:49:14.849761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:14.849720    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45459602-ee7f-4e32-8f09-41106c0275a5" path="/var/lib/kubelet/pods/45459602-ee7f-4e32-8f09-41106c0275a5/volumes"
Oct 30 21:49:16.339568 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:16.339530    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:49:19.161744 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:19.161712    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:49:19.162228 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:19.161785    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:49:33.650332 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:33.650296    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"]
Oct 30 21:49:33.650964 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:33.650533    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x" podUID="4826538a-d05c-46ad-ad06-b693beec405f" containerName="maas-api" containerID="cri-o://0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5" gracePeriod=30
Oct 30 21:49:33.964432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:33.964400    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:49:34.087226 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.087181    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4826538a-d05c-46ad-ad06-b693beec405f-kube-api-access-c72md\" (UniqueName: \"kubernetes.io/projected/4826538a-d05c-46ad-ad06-b693beec405f-kube-api-access-c72md\") pod \"4826538a-d05c-46ad-ad06-b693beec405f\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") "
Oct 30 21:49:34.087466 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.087328    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4826538a-d05c-46ad-ad06-b693beec405f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4826538a-d05c-46ad-ad06-b693beec405f-openshift-service-ca\") pod \"4826538a-d05c-46ad-ad06-b693beec405f\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") "
Oct 30 21:49:34.087466 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.087399    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-maas-api-tls\") pod \"4826538a-d05c-46ad-ad06-b693beec405f\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") "
Oct 30 21:49:34.087466 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.087434    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-metrics-certs\") pod \"4826538a-d05c-46ad-ad06-b693beec405f\" (UID: \"4826538a-d05c-46ad-ad06-b693beec405f\") "
Oct 30 21:49:34.088055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.088018    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4826538a-d05c-46ad-ad06-b693beec405f-openshift-service-ca" pod "4826538a-d05c-46ad-ad06-b693beec405f" (UID: "4826538a-d05c-46ad-ad06-b693beec405f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:34.088335 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.088308    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4826538a-d05c-46ad-ad06-b693beec405f-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.091054 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.091028    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-maas-api-tls" pod "4826538a-d05c-46ad-ad06-b693beec405f" (UID: "4826538a-d05c-46ad-ad06-b693beec405f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:34.091182 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.091082    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4826538a-d05c-46ad-ad06-b693beec405f-kube-api-access-c72md" pod "4826538a-d05c-46ad-ad06-b693beec405f" (UID: "4826538a-d05c-46ad-ad06-b693beec405f"). InnerVolumeSpecName "kube-api-access-c72md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:34.091306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.091244    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-metrics-certs" pod "4826538a-d05c-46ad-ad06-b693beec405f" (UID: "4826538a-d05c-46ad-ad06-b693beec405f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:34.189631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.189553    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.189631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.189583    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4826538a-d05c-46ad-ad06-b693beec405f-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.189631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.189593    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c72md\" (UniqueName: \"kubernetes.io/projected/4826538a-d05c-46ad-ad06-b693beec405f-kube-api-access-c72md\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:34.346773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.346740    5817 generic.go:356] "Generic (PLEG): container finished" podID="4826538a-d05c-46ad-ad06-b693beec405f" containerID="0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5" exitCode=0
Oct 30 21:49:34.346978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.346783    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"
Oct 30 21:49:34.346978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.346820    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x" event={"ID":"4826538a-d05c-46ad-ad06-b693beec405f","Type":"ContainerDied","Data":"0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5"}
Oct 30 21:49:34.346978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.346884    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x" event={"ID":"4826538a-d05c-46ad-ad06-b693beec405f","Type":"ContainerDied","Data":"28fac79c4c5e431bb26d22ed3e1555b4314c79c42bad26417cd28ec20e62ac4e"}
Oct 30 21:49:34.346978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.346916    5817 scope.go:122] "RemoveContainer" containerID="0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5"
Oct 30 21:49:34.359083 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.359066    5817 scope.go:122] "RemoveContainer" containerID="0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5"
Oct 30 21:49:34.359409 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:34.359387    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5\": container with ID starting with 0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5 not found: ID does not exist" containerID="0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5"
Oct 30 21:49:34.359499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.359420    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5"} err="failed to get container status \"0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5\": rpc error: code = NotFound desc = could not find container \"0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5\": container with ID starting with 0375a93a115a7ec32bfcaa795fb4c6d10d6a1c3904764c91aaf04d5ce7b4fec5 not found: ID does not exist"
Oct 30 21:49:34.514961 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.514898    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"]
Oct 30 21:49:34.520218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.520194    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af87dbaf-6c8c59b758-fk56x"]
Oct 30 21:49:34.845524 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:34.845442    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4826538a-d05c-46ad-ad06-b693beec405f" path="/var/lib/kubelet/pods/4826538a-d05c-46ad-ad06-b693beec405f/volumes"
Oct 30 21:49:36.357492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:36.357454    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"]
Oct 30 21:49:36.357908 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:36.357778    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy" containerID="cri-o://0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6" gracePeriod=30
Oct 30 21:49:41.648832 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.648798    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:49:41.711987 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.711958    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-istio-token\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-istio-token\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712040    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-socket\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712194    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-credential-socket\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712228    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81ca29db-8554-4880-877a-78576d6e0b78-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca29db-8554-4880-877a-78576d6e0b78-istiod-ca-cert\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712266    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-certs\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712295    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-envoy\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712312    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-socket" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:41.712383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712324    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81ca29db-8554-4880-877a-78576d6e0b78-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca29db-8554-4880-877a-78576d6e0b78-istio-podinfo\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712446    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-kube-api-access-zc664\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-kube-api-access-zc664\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712485    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-data\") pod \"81ca29db-8554-4880-877a-78576d6e0b78\" (UID: \"81ca29db-8554-4880-877a-78576d6e0b78\") "
Oct 30 21:49:41.712588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712446    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-credential-socket" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:41.712588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712549    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-certs" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:41.712777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712721    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-data" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:41.712777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.712733    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81ca29db-8554-4880-877a-78576d6e0b78-istiod-ca-cert" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:41.713256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.713222    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.713256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.713246    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81ca29db-8554-4880-877a-78576d6e0b78-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.713493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.713260    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.713493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.713273    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.713493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.713282    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.714753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.714730    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81ca29db-8554-4880-877a-78576d6e0b78-istio-podinfo" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:49:41.714834 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.714782    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-istio-token" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:41.714980 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.714966    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-envoy" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:41.715078 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.715061    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-kube-api-access-zc664" pod "81ca29db-8554-4880-877a-78576d6e0b78" (UID: "81ca29db-8554-4880-877a-78576d6e0b78"). InnerVolumeSpecName "kube-api-access-zc664". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:41.814498 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.814470    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81ca29db-8554-4880-877a-78576d6e0b78-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.814498 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.814499    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81ca29db-8554-4880-877a-78576d6e0b78-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.814711 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.814513    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc664\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-kube-api-access-zc664\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:41.814711 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:41.814527    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81ca29db-8554-4880-877a-78576d6e0b78-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:42.405815 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.405782    5817 generic.go:356] "Generic (PLEG): container finished" podID="81ca29db-8554-4880-877a-78576d6e0b78" containerID="0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6" exitCode=0
Oct 30 21:49:42.406021 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.405839    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"
Oct 30 21:49:42.406021 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.405865    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" event={"ID":"81ca29db-8554-4880-877a-78576d6e0b78","Type":"ContainerDied","Data":"0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6"}
Oct 30 21:49:42.406021 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.405919    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h" event={"ID":"81ca29db-8554-4880-877a-78576d6e0b78","Type":"ContainerDied","Data":"a58b39aa02dfc894a0a36dee55ba832074a2c9bc861af0cea512d2434a2e77d7"}
Oct 30 21:49:42.406021 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.405942    5817 scope.go:122] "RemoveContainer" containerID="0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6"
Oct 30 21:49:42.419665 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.419646    5817 scope.go:122] "RemoveContainer" containerID="1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c"
Oct 30 21:49:42.438688 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.438428    5817 scope.go:122] "RemoveContainer" containerID="0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6"
Oct 30 21:49:42.438932 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:42.438903    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6\": container with ID starting with 0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6 not found: ID does not exist" containerID="0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6"
Oct 30 21:49:42.439035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.438945    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6"} err="failed to get container status \"0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6\": rpc error: code = NotFound desc = could not find container \"0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6\": container with ID starting with 0fafaea745bd1cc42e7c8656e508ab36ef7e3447d1cccd172b5ca7eb6733eeb6 not found: ID does not exist"
Oct 30 21:49:42.439035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.438973    5817 scope.go:122] "RemoveContainer" containerID="1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c"
Oct 30 21:49:42.439471 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:42.439436    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c\": container with ID starting with 1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c not found: ID does not exist" containerID="1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c"
Oct 30 21:49:42.439563 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.439480    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c"} err="failed to get container status \"1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c\": rpc error: code = NotFound desc = could not find container \"1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c\": container with ID starting with 1da73512f3881a9d723fe71c8a240f17c061d8097689060e92b44640a2cb105c not found: ID does not exist"
Oct 30 21:49:42.610792 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.610762    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"]
Oct 30 21:49:42.658180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.658103    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-af87dbaf-openshift-default-67d6b99cdc-dtd6h"]
Oct 30 21:49:42.844863 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:42.844834    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81ca29db-8554-4880-877a-78576d6e0b78" path="/var/lib/kubelet/pods/81ca29db-8554-4880-877a-78576d6e0b78/volumes"
Oct 30 21:49:45.623886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.623847    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"]
Oct 30 21:49:45.624790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624765    5817 cpu_manager.go:414] "containerMap: removing container" podUID="4826538a-d05c-46ad-ad06-b693beec405f" containerName="maas-api"
Oct 30 21:49:45.624790 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624788    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4826538a-d05c-46ad-ad06-b693beec405f" containerName="maas-api"
Oct 30 21:49:45.624955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624824    5817 cpu_manager.go:414] "containerMap: removing container" podUID="45459602-ee7f-4e32-8f09-41106c0275a5" containerName="maas-api"
Oct 30 21:49:45.624955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624829    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45459602-ee7f-4e32-8f09-41106c0275a5" containerName="maas-api"
Oct 30 21:49:45.624955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624851    5817 cpu_manager.go:414] "containerMap: removing container" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy"
Oct 30 21:49:45.624955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624857    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy"
Oct 30 21:49:45.624955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624866    5817 cpu_manager.go:414] "containerMap: removing container" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy"
Oct 30 21:49:45.624955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624871    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy"
Oct 30 21:49:45.625138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624972    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="45459602-ee7f-4e32-8f09-41106c0275a5" containerName="maas-api"
Oct 30 21:49:45.625138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624985    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy"
Oct 30 21:49:45.625138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.624991    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="81ca29db-8554-4880-877a-78576d6e0b78" containerName="istio-proxy"
Oct 30 21:49:45.625138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.625000    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="4826538a-d05c-46ad-ad06-b693beec405f" containerName="maas-api"
Oct 30 21:49:45.628999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.628981    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.687246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.687205    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bfdf30b9-openshift-default-dockercfg-pzmxv\""
Oct 30 21:49:45.748817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.748774    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"]
Oct 30 21:49:45.825036 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.824997    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825228 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825055    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825314 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825284    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd7217ba-460e-45c6-804b-70ebd50c3d38-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825410 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825388    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825531 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825504    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825563    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdcw9\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-kube-api-access-qdcw9\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825782    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825858    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.825920 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.825904    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927037 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.926914    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd7217ba-460e-45c6-804b-70ebd50c3d38-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927037 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.926988    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927309 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927085    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927309 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927143    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdcw9\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-kube-api-access-qdcw9\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927309 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927302    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927395    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927437    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927470    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927493    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927529    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927820 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927733    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd7217ba-460e-45c6-804b-70ebd50c3d38-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.927946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.927921    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.928066 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.928009    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.928554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.928527    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.930198 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.930179    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.930770 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.930748    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.987596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.987560    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdcw9\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-kube-api-access-qdcw9\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:45.988162 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:45.988069    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:46.242419 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:46.242389    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:46.359509 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:46.359466    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:49:46.713337 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:46.713296    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"]
Oct 30 21:49:46.742650 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:49:46.742622    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7217ba_460e_45c6_804b_70ebd50c3d38.slice/crio-f80753d6454d5955d44696ca2b278c14c80f2a2a610ccc8e4a1281f755330e72 WatchSource:0}: Error finding container f80753d6454d5955d44696ca2b278c14c80f2a2a610ccc8e4a1281f755330e72: Status 404 returned error can't find the container with id f80753d6454d5955d44696ca2b278c14c80f2a2a610ccc8e4a1281f755330e72
Oct 30 21:49:46.745232 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:46.745197    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:46.745308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:46.745269    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:46.745388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:46.745307    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:49:47.455675 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:47.455640    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" event={"ID":"fd7217ba-460e-45c6-804b-70ebd50c3d38","Type":"ContainerStarted","Data":"5ca32526f9bb2988d78ae56c0e7cd218e8caf1cdc2545a58cc91e8e12b4c1d99"}
Oct 30 21:49:47.455859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:47.455703    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" event={"ID":"fd7217ba-460e-45c6-804b-70ebd50c3d38","Type":"ContainerStarted","Data":"f80753d6454d5955d44696ca2b278c14c80f2a2a610ccc8e4a1281f755330e72"}
Oct 30 21:49:47.621038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:47.620974    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podStartSLOduration=2.620957557 podStartE2EDuration="2.620957557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:47.617865093 +0000 UTC m=+2122.905587639" watchObservedRunningTime="2026-10-30 21:49:47.620957557 +0000 UTC m=+2122.908680103"
Oct 30 21:49:48.243587 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:48.243558    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:48.249447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:48.249418    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:48.462222 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:48.462184    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:48.463248 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:48.463224    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:49:49.791799 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.791766    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"]
Oct 30 21:49:49.796545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.796520    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.853053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.853023    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-fbdbd87a-serving-cert\""
Oct 30 21:49:49.886459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.886420    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.886645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.886487    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c353ed2-61e8-46fd-a388-8d337921f529-openshift-service-ca\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.886645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.886547    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5g2v\" (UniqueName: \"kubernetes.io/projected/2c353ed2-61e8-46fd-a388-8d337921f529-kube-api-access-j5g2v\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.886931 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.886902    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-metrics-certs\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.904556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.904524    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"]
Oct 30 21:49:49.988099 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.988058    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-metrics-certs\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.988245 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.988160    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.988245 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.988199    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c353ed2-61e8-46fd-a388-8d337921f529-openshift-service-ca\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.988245 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.988223    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5g2v\" (UniqueName: \"kubernetes.io/projected/2c353ed2-61e8-46fd-a388-8d337921f529-kube-api-access-j5g2v\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.988482 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:49.988306    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-serving-cert: secret "maas-api-e2e-migrate-fbdbd87a-serving-cert" not found
Oct 30 21:49:49.988482 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:49:49.988413    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls podName:2c353ed2-61e8-46fd-a388-8d337921f529 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:50.488394837 +0000 UTC m=+2125.776117383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls") pod "maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8" (UID: "2c353ed2-61e8-46fd-a388-8d337921f529") : secret "maas-api-e2e-migrate-fbdbd87a-serving-cert" not found
Oct 30 21:49:49.988883 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.988860    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c353ed2-61e8-46fd-a388-8d337921f529-openshift-service-ca\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:49.990588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:49.990565    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-metrics-certs\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:50.047801 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:50.047723    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5g2v\" (UniqueName: \"kubernetes.io/projected/2c353ed2-61e8-46fd-a388-8d337921f529-kube-api-access-j5g2v\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:50.496014 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:50.495979    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:50.498450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:50.498426    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls\") pod \"maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:50.712447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:50.712405    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:51.090638 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:51.090585    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"]
Oct 30 21:49:51.091074 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:49:51.090888    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c353ed2_61e8_46fd_a388_8d337921f529.slice/crio-4ff6249eab69833fe21617655949fd7aef004746e503d004e182404504633b22 WatchSource:0}: Error finding container 4ff6249eab69833fe21617655949fd7aef004746e503d004e182404504633b22: Status 404 returned error can't find the container with id 4ff6249eab69833fe21617655949fd7aef004746e503d004e182404504633b22
Oct 30 21:49:51.486150 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:51.486114    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8" event={"ID":"2c353ed2-61e8-46fd-a388-8d337921f529","Type":"ContainerStarted","Data":"4ff6249eab69833fe21617655949fd7aef004746e503d004e182404504633b22"}
Oct 30 21:49:52.494206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:52.494171    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8" event={"ID":"2c353ed2-61e8-46fd-a388-8d337921f529","Type":"ContainerStarted","Data":"e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b"}
Oct 30 21:49:52.494603 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:52.494239    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:49:52.608004 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:52.607948    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8" podStartSLOduration=2.738072292 podStartE2EDuration="3.607931595s" totalImagesPullingTime="869.859303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:52.605040051 +0000 UTC m=+2127.892762597" watchObservedRunningTime="2026-10-30 21:49:52.607931595 +0000 UTC m=+2127.895654188"
Oct 30 21:49:54.074458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.074385    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"]
Oct 30 21:49:54.079571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.079547    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.129390 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.129360    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bfdf30b9\""
Oct 30 21:49:54.175998 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.175967    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bfdf30b9-dockercfg-5h8cl\""
Oct 30 21:49:54.180652 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.180621    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"]
Oct 30 21:49:54.198793 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.198752    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrrjd\" (UniqueName: \"kubernetes.io/projected/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-kube-api-access-xrrjd\") pod \"payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.199250 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.199224    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.282125 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.282086    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"]
Oct 30 21:49:54.288134 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.288110    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"]
Oct 30 21:49:54.288473 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.288453    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.301388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.301316    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrrjd\" (UniqueName: \"kubernetes.io/projected/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-kube-api-access-xrrjd\") pod \"payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.301556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.301535    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.302437 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.302414    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.362721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.362642    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrrjd\" (UniqueName: \"kubernetes.io/projected/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-kube-api-access-xrrjd\") pod \"payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.396965 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.396926    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:54.403802 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.403765    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99659e40-3d4e-42ad-83a6-310398106827-plugins-config-volume\") pod \"payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") " pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.403963 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.403942    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlr9z\" (UniqueName: \"kubernetes.io/projected/99659e40-3d4e-42ad-83a6-310398106827-kube-api-access-jlr9z\") pod \"payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") " pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.517074 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.515819    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlr9z\" (UniqueName: \"kubernetes.io/projected/99659e40-3d4e-42ad-83a6-310398106827-kube-api-access-jlr9z\") pod \"payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") " pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.517074 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.516045    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99659e40-3d4e-42ad-83a6-310398106827-plugins-config-volume\") pod \"payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") " pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.517074 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.516945    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99659e40-3d4e-42ad-83a6-310398106827-plugins-config-volume\") pod \"payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") " pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.574963 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.574929    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlr9z\" (UniqueName: \"kubernetes.io/projected/99659e40-3d4e-42ad-83a6-310398106827-kube-api-access-jlr9z\") pod \"payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") " pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.614242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.614164    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:54.803429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.803404    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"]
Oct 30 21:49:54.805635 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:49:54.805605    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a4a5ad_8af5_4a47_8b77_1ffa94872da0.slice/crio-b0969e5b4d0268429031b6eb1cdb2075e39358de5349701c3d44265f0956b925 WatchSource:0}: Error finding container b0969e5b4d0268429031b6eb1cdb2075e39358de5349701c3d44265f0956b925: Status 404 returned error can't find the container with id b0969e5b4d0268429031b6eb1cdb2075e39358de5349701c3d44265f0956b925
Oct 30 21:49:54.996908 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:54.996884    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"]
Oct 30 21:49:54.997033 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:49:54.996888    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99659e40_3d4e_42ad_83a6_310398106827.slice/crio-dc284ef2ecc12990194d7018430cfd930abef4b817ccff0365b62c92b19cda43 WatchSource:0}: Error finding container dc284ef2ecc12990194d7018430cfd930abef4b817ccff0365b62c92b19cda43: Status 404 returned error can't find the container with id dc284ef2ecc12990194d7018430cfd930abef4b817ccff0365b62c92b19cda43
Oct 30 21:49:55.536268 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.536231    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" event={"ID":"14a4a5ad-8af5-4a47-8b77-1ffa94872da0","Type":"ContainerStarted","Data":"aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277"}
Oct 30 21:49:55.536697 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.536293    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" event={"ID":"14a4a5ad-8af5-4a47-8b77-1ffa94872da0","Type":"ContainerStarted","Data":"b0969e5b4d0268429031b6eb1cdb2075e39358de5349701c3d44265f0956b925"}
Oct 30 21:49:55.536697 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.536335    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:49:55.538038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.538010    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" event={"ID":"99659e40-3d4e-42ad-83a6-310398106827","Type":"ContainerStarted","Data":"10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556"}
Oct 30 21:49:55.538144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.538061    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" event={"ID":"99659e40-3d4e-42ad-83a6-310398106827","Type":"ContainerStarted","Data":"dc284ef2ecc12990194d7018430cfd930abef4b817ccff0365b62c92b19cda43"}
Oct 30 21:49:55.538144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.538083    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:49:55.647572 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.647522    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" podStartSLOduration=1.6475089010000001 podStartE2EDuration="1.647508901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:55.645837323 +0000 UTC m=+2130.933559867" watchObservedRunningTime="2026-10-30 21:49:55.647508901 +0000 UTC m=+2130.935231446"
Oct 30 21:49:55.760060 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:55.759654    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" podStartSLOduration=1.759635287 podStartE2EDuration="1.759635287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:55.756413129 +0000 UTC m=+2131.044135677" watchObservedRunningTime="2026-10-30 21:49:55.759635287 +0000 UTC m=+2131.047357843"
Oct 30 21:49:58.507796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:49:58.507763    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:50:04.345593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.345556    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"]
Oct 30 21:50:04.350991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.350964    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374331 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374293    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374564 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374434    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374594    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5xmd\" (UniqueName: \"kubernetes.io/projected/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-kube-api-access-n5xmd\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374685 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374634    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374802 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374728    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374802 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374753    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374912 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374805    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374912 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374882    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.374912 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.374905    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.452904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.452863    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"]
Oct 30 21:50:04.475996 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.475967    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5xmd\" (UniqueName: \"kubernetes.io/projected/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-kube-api-access-n5xmd\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476178 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476010    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476178 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476166    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476196    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476254    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476333    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476378    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476543 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476502    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476600 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476553    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476658 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476637    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.476974 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476947    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.477076 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.476997    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.477201 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.477174    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.477968 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.477939    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.479239 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.479213    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.495368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.494310    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.535242 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.535210    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5xmd\" (UniqueName: \"kubernetes.io/projected/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-kube-api-access-n5xmd\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.535971 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.535950    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c1c929f-9bf9-43f3-bf99-3c0d913b1913-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-s22vv\" (UID: \"9c1c929f-9bf9-43f3-bf99-3c0d913b1913\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:04.671515 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:04.671435    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:05.108597 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.108562    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"]
Oct 30 21:50:05.109615 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:50:05.109587    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1c929f_9bf9_43f3_bf99_3c0d913b1913.slice/crio-a05e364c266ab6511b47a9f306dd6e47ef161261335e139d4a5d3682a25dff9d WatchSource:0}: Error finding container a05e364c266ab6511b47a9f306dd6e47ef161261335e139d4a5d3682a25dff9d: Status 404 returned error can't find the container with id a05e364c266ab6511b47a9f306dd6e47ef161261335e139d4a5d3682a25dff9d
Oct 30 21:50:05.112161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.112124    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:05.112246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.112191    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:05.112246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.112220    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:05.617152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.617109    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" event={"ID":"9c1c929f-9bf9-43f3-bf99-3c0d913b1913","Type":"ContainerStarted","Data":"68eda447e135f6e659f953da6ffa4bce3f96edadee0fcd08564280f2dead8c0b"}
Oct 30 21:50:05.617623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.617176    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" event={"ID":"9c1c929f-9bf9-43f3-bf99-3c0d913b1913","Type":"ContainerStarted","Data":"a05e364c266ab6511b47a9f306dd6e47ef161261335e139d4a5d3682a25dff9d"}
Oct 30 21:50:05.672048 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.672014    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:05.735611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:05.735561    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podStartSLOduration=1.735545985 podStartE2EDuration="1.735545985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:05.733610685 +0000 UTC m=+2141.021333231" watchObservedRunningTime="2026-10-30 21:50:05.735545985 +0000 UTC m=+2141.023268532"
Oct 30 21:50:06.548753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:06.548724    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:50:06.548918 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:06.548793    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:50:06.673893 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:06.673858    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-s22vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 21:50:06.674298 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:06.673917    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podUID="9c1c929f-9bf9-43f3-bf99-3c0d913b1913" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 21:50:07.671824 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:07.671786    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-s22vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 21:50:07.671998 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:07.671837    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podUID="9c1c929f-9bf9-43f3-bf99-3c0d913b1913" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 21:50:08.671846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:08.671814    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-s22vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 21:50:08.672374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:08.671880    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podUID="9c1c929f-9bf9-43f3-bf99-3c0d913b1913" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 21:50:09.672550 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:09.672515    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-s22vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 21:50:09.673093 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:09.672578    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podUID="9c1c929f-9bf9-43f3-bf99-3c0d913b1913" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 21:50:10.679029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:10.678994    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-s22vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 21:50:10.679759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:10.679722    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podUID="9c1c929f-9bf9-43f3-bf99-3c0d913b1913" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 21:50:11.672340 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:11.672304    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-s22vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 21:50:11.672537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:11.672383    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podUID="9c1c929f-9bf9-43f3-bf99-3c0d913b1913" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 21:50:12.671817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:12.671780    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-s22vv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 21:50:12.672304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:12.671859    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv" podUID="9c1c929f-9bf9-43f3-bf99-3c0d913b1913" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 21:50:14.664472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:14.664440    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:14.664861 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:14.664842    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:14.665587 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:14.665566    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-s22vv"
Oct 30 21:50:16.356296 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:50:16.356251    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:50:18.114645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:18.114586    5817 patch_prober.go:29] interesting pod/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:50:18.115135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:18.114682    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:50:19.479646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.479609    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"]
Oct 30 21:50:19.507210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.507176    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.613978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.613940    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"]
Oct 30 21:50:19.671435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.671386    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.671644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.671452    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.671644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.671523    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmx9x\" (UniqueName: \"kubernetes.io/projected/be00a1a8-0b75-48fa-8863-c2150f20d975-kube-api-access-gmx9x\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.671853 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.671787    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.671940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.671895    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.672020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.671991    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be00a1a8-0b75-48fa-8863-c2150f20d975-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.672122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.672107    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.672180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.672136    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.672234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.672186    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774117 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774019    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774117 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774066    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774219    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmx9x\" (UniqueName: \"kubernetes.io/projected/be00a1a8-0b75-48fa-8863-c2150f20d975-kube-api-access-gmx9x\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774387    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774555 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774435    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774555 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774523    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be00a1a8-0b75-48fa-8863-c2150f20d975-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774559    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774733 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774711    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774794 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774756    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774832    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774910 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774854    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.774910 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.774880    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.775545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.775197    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.775545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.775242    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be00a1a8-0b75-48fa-8863-c2150f20d975-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.776995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.776970    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.777882 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.777848    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.833316 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.833281    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be00a1a8-0b75-48fa-8863-c2150f20d975-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.833484 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.833366    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmx9x\" (UniqueName: \"kubernetes.io/projected/be00a1a8-0b75-48fa-8863-c2150f20d975-kube-api-access-gmx9x\") pod \"maas-default-gateway-openshift-default-67bf986f99-tr966\" (UID: \"be00a1a8-0b75-48fa-8863-c2150f20d975\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:19.841635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:19.841610    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:20.293579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.293515    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"]
Oct 30 21:50:20.326604 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:50:20.326576    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe00a1a8_0b75_48fa_8863_c2150f20d975.slice/crio-6c551ef1c28244d97abe5830780d1621966b1b9283a679ac11f41c693d9f2c8a WatchSource:0}: Error finding container 6c551ef1c28244d97abe5830780d1621966b1b9283a679ac11f41c693d9f2c8a: Status 404 returned error can't find the container with id 6c551ef1c28244d97abe5830780d1621966b1b9283a679ac11f41c693d9f2c8a
Oct 30 21:50:20.329885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.329834    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:20.329964 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.329942    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:20.330015 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.329997    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:20.794894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.794858    5817 generic.go:356] "Generic (PLEG): container finished" podID="bace0d0d-712e-4138-88da-98c90e39413a" containerID="4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02" exitCode=0
Oct 30 21:50:20.795319 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.794926    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" event={"ID":"bace0d0d-712e-4138-88da-98c90e39413a","Type":"ContainerDied","Data":"4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02"}
Oct 30 21:50:20.795404 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.795329    5817 scope.go:122] "RemoveContainer" containerID="4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02"
Oct 30 21:50:20.797125 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.797103    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" event={"ID":"be00a1a8-0b75-48fa-8863-c2150f20d975","Type":"ContainerStarted","Data":"9b6a73b7600401ff620db5bc01ae297bc78eb0c7f31ef0064b31acaf7826d609"}
Oct 30 21:50:20.797234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.797148    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" event={"ID":"be00a1a8-0b75-48fa-8863-c2150f20d975","Type":"ContainerStarted","Data":"6c551ef1c28244d97abe5830780d1621966b1b9283a679ac11f41c693d9f2c8a"}
Oct 30 21:50:20.797234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.797145    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:20.797234 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.797219    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:20.797338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.797260    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:50:20.846263 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:20.846222    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:21.059676 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:21.059565    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podStartSLOduration=2.059549499 podStartE2EDuration="2.059549499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:21.057710236 +0000 UTC m=+2156.345432776" watchObservedRunningTime="2026-10-30 21:50:21.059549499 +0000 UTC m=+2156.347272044"
Oct 30 21:50:21.851547 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:21.851504    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 21:50:21.852087 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:21.851565    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 21:50:21.856633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:21.856597    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" event={"ID":"bace0d0d-712e-4138-88da-98c90e39413a","Type":"ContainerStarted","Data":"395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d"}
Oct 30 21:50:22.013369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:22.010809    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:50:22.013369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:22.010947    5817 patch_prober.go:29] interesting pod/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:50:22.013369 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:22.010991    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:50:22.847110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:22.845715    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 21:50:22.847110 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:22.845774    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 21:50:22.996723 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:22.996684    5817 patch_prober.go:29] interesting pod/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:50:23.024637 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:23.024585    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:50:23.842324 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:23.842271    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 21:50:23.842559 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:23.842367    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 21:50:24.010817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:24.010777    5817 patch_prober.go:29] interesting pod/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:50:24.011431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:24.011396    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:50:24.843524 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:24.843483    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 21:50:24.843794 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:24.843764    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 21:50:25.001370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:24.996843    5817 patch_prober.go:29] interesting pod/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:50:25.001370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:24.996907    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:50:25.843730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:25.843228    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 21:50:25.843730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:25.843296    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 21:50:26.012221 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:26.012185    5817 patch_prober.go:29] interesting pod/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:50:26.012524 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:26.012492    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:50:26.842044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:26.842001    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 21:50:26.842227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:26.842083    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 21:50:26.997072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:26.997026    5817 patch_prober.go:29] interesting pod/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:50:26.997501 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:26.997099    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:50:27.842772 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:27.842737    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 30 21:50:27.842948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:27.842790    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 30 21:50:28.001015 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:28.000984    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:50:28.001472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:28.001455    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:50:28.002422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:28.002404    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:50:29.116579 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:29.116531    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:29.117071 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:29.117020    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:29.118100 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:29.118077    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966"
Oct 30 21:50:37.116184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.116104    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"]
Oct 30 21:50:37.120878 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.120855    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.141565 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.141543    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-metrics-certs\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.141741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.141718    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvfqt\" (UniqueName: \"kubernetes.io/projected/a52eda1a-e83b-4967-9410-15b96a743607-kube-api-access-bvfqt\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.141834 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.141822    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.141891 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.141859    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52eda1a-e83b-4967-9410-15b96a743607-openshift-service-ca\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.219033 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.219000    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bfdf30b9-serving-cert\""
Oct 30 21:50:37.223113 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.223082    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"]
Oct 30 21:50:37.243535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.243506    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-metrics-certs\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.243822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.243797    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvfqt\" (UniqueName: \"kubernetes.io/projected/a52eda1a-e83b-4967-9410-15b96a743607-kube-api-access-bvfqt\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.243944 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.243927    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.244095 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:50:37.244071    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-serving-cert: secret "maas-api-e2e-mt-bfdf30b9-serving-cert" not found
Oct 30 21:50:37.244171 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:50:37.244157    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls podName:a52eda1a-e83b-4967-9410-15b96a743607 nodeName:}" failed. No retries permitted until 2026-10-30 21:50:37.744137155 +0000 UTC m=+2173.031859702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls") pod "maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7" (UID: "a52eda1a-e83b-4967-9410-15b96a743607") : secret "maas-api-e2e-mt-bfdf30b9-serving-cert" not found
Oct 30 21:50:37.244262 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.244075    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52eda1a-e83b-4967-9410-15b96a743607-openshift-service-ca\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.244886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.244863    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52eda1a-e83b-4967-9410-15b96a743607-openshift-service-ca\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.247037 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.247016    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-metrics-certs\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.329724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.329697    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvfqt\" (UniqueName: \"kubernetes.io/projected/a52eda1a-e83b-4967-9410-15b96a743607-kube-api-access-bvfqt\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.760695 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.760657    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:37.763231 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:37.763206    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls\") pod \"maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:38.034759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:38.034664    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:38.411075 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:38.411042    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"]
Oct 30 21:50:38.411474 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:50:38.411292    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52eda1a_e83b_4967_9410_15b96a743607.slice/crio-a7df7b9a3f6b227dd8e99b9a3ed1885da45b376fec25250369ba46704b14d838 WatchSource:0}: Error finding container a7df7b9a3f6b227dd8e99b9a3ed1885da45b376fec25250369ba46704b14d838: Status 404 returned error can't find the container with id a7df7b9a3f6b227dd8e99b9a3ed1885da45b376fec25250369ba46704b14d838
Oct 30 21:50:38.412811 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:38.412790    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:50:38.998148 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:38.998115    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7" event={"ID":"a52eda1a-e83b-4967-9410-15b96a743607","Type":"ContainerStarted","Data":"a7df7b9a3f6b227dd8e99b9a3ed1885da45b376fec25250369ba46704b14d838"}
Oct 30 21:50:39.051191 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:39.051145    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"]
Oct 30 21:50:39.098088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:39.098055    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-27e86670-8bvx8"]
Oct 30 21:50:40.007578 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:40.007540    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7" event={"ID":"a52eda1a-e83b-4967-9410-15b96a743607","Type":"ContainerStarted","Data":"41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283"}
Oct 30 21:50:40.007983 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:40.007616    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:40.845607 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:40.845575    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="752c7fd7-d9e3-4fcd-b858-61de482508ec" path="/var/lib/kubelet/pods/752c7fd7-d9e3-4fcd-b858-61de482508ec/volumes"
Oct 30 21:50:46.019995 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:46.019964    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:50:46.134254 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:50:46.134206    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7" podStartSLOduration=8.194574996 podStartE2EDuration="9.134191576s" totalImagesPullingTime="939.61658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:40.133544601 +0000 UTC m=+2175.421267149" watchObservedRunningTime="2026-10-30 21:50:46.134191576 +0000 UTC m=+2181.421914122"
Oct 30 21:50:46.339808 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:50:46.339724    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:51:03.346080 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:03.345924    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"]
Oct 30 21:51:03.468112 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:03.468075    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"]
Oct 30 21:51:08.346627 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.346583    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7" podUID="a4ab6413-5433-495d-b60c-bfb93a7b58df" containerName="payload-processing" containerID="cri-o://46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0" gracePeriod=25
Oct 30 21:51:08.468389 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.468357    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2" podUID="80ee7ba8-9f02-4ac1-af48-02e29f24f82d" containerName="payload-pre-processing" containerID="cri-o://ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8" gracePeriod=25
Oct 30 21:51:08.636307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.636275    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:51:08.733650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.733598    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4ab6413-5433-495d-b60c-bfb93a7b58df-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ab6413-5433-495d-b60c-bfb93a7b58df-plugins-config-volume\") pod \"a4ab6413-5433-495d-b60c-bfb93a7b58df\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") "
Oct 30 21:51:08.733850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.733705    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4ab6413-5433-495d-b60c-bfb93a7b58df-kube-api-access-99r6q\" (UniqueName: \"kubernetes.io/projected/a4ab6413-5433-495d-b60c-bfb93a7b58df-kube-api-access-99r6q\") pod \"a4ab6413-5433-495d-b60c-bfb93a7b58df\" (UID: \"a4ab6413-5433-495d-b60c-bfb93a7b58df\") "
Oct 30 21:51:08.734082 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.734051    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ab6413-5433-495d-b60c-bfb93a7b58df-plugins-config-volume" pod "a4ab6413-5433-495d-b60c-bfb93a7b58df" (UID: "a4ab6413-5433-495d-b60c-bfb93a7b58df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:08.734682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.734659    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ab6413-5433-495d-b60c-bfb93a7b58df-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:08.736273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.736245    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ab6413-5433-495d-b60c-bfb93a7b58df-kube-api-access-99r6q" pod "a4ab6413-5433-495d-b60c-bfb93a7b58df" (UID: "a4ab6413-5433-495d-b60c-bfb93a7b58df"). InnerVolumeSpecName "kube-api-access-99r6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:08.762356 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.762312    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:51:08.835190 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.835151    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-plugins-config-volume\") pod \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") "
Oct 30 21:51:08.835439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.835230    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-kube-api-access-t7hsg\" (UniqueName: \"kubernetes.io/projected/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-kube-api-access-t7hsg\") pod \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\" (UID: \"80ee7ba8-9f02-4ac1-af48-02e29f24f82d\") "
Oct 30 21:51:08.835683 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.835643    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-plugins-config-volume" pod "80ee7ba8-9f02-4ac1-af48-02e29f24f82d" (UID: "80ee7ba8-9f02-4ac1-af48-02e29f24f82d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:08.836176 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.836157    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:08.836273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.836180    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99r6q\" (UniqueName: \"kubernetes.io/projected/a4ab6413-5433-495d-b60c-bfb93a7b58df-kube-api-access-99r6q\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:08.837565 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.837543    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-kube-api-access-t7hsg" pod "80ee7ba8-9f02-4ac1-af48-02e29f24f82d" (UID: "80ee7ba8-9f02-4ac1-af48-02e29f24f82d"). InnerVolumeSpecName "kube-api-access-t7hsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:08.938489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:08.938451    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7hsg\" (UniqueName: \"kubernetes.io/projected/80ee7ba8-9f02-4ac1-af48-02e29f24f82d-kube-api-access-t7hsg\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:09.262056 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.262009    5817 generic.go:356] "Generic (PLEG): container finished" podID="a4ab6413-5433-495d-b60c-bfb93a7b58df" containerID="46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0" exitCode=0
Oct 30 21:51:09.262256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.262060    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"
Oct 30 21:51:09.262256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.262155    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7" event={"ID":"a4ab6413-5433-495d-b60c-bfb93a7b58df","Type":"ContainerDied","Data":"46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0"}
Oct 30 21:51:09.262256 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.262224    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7" event={"ID":"a4ab6413-5433-495d-b60c-bfb93a7b58df","Type":"ContainerDied","Data":"9476eaee9fb687bf28b4e4c5a5d998529c589a9825dfe83c2d56ecaf40519878"}
Oct 30 21:51:09.262450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.262269    5817 scope.go:122] "RemoveContainer" containerID="46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0"
Oct 30 21:51:09.264302 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.264277    5817 generic.go:356] "Generic (PLEG): container finished" podID="80ee7ba8-9f02-4ac1-af48-02e29f24f82d" containerID="ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8" exitCode=0
Oct 30 21:51:09.264459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.264406    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2" event={"ID":"80ee7ba8-9f02-4ac1-af48-02e29f24f82d","Type":"ContainerDied","Data":"ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8"}
Oct 30 21:51:09.264459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.264449    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2" event={"ID":"80ee7ba8-9f02-4ac1-af48-02e29f24f82d","Type":"ContainerDied","Data":"a9bb4b3c797df530c188ca146eb09f1f9c989c1b7ec2edcaac7508821a89373c"}
Oct 30 21:51:09.264576 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.264516    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"
Oct 30 21:51:09.273546 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.273522    5817 scope.go:122] "RemoveContainer" containerID="46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0"
Oct 30 21:51:09.273796 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:51:09.273777    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0\": container with ID starting with 46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0 not found: ID does not exist" containerID="46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0"
Oct 30 21:51:09.273854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.273803    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0"} err="failed to get container status \"46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0\": rpc error: code = NotFound desc = could not find container \"46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0\": container with ID starting with 46372613d757414c1ceed16ae25bff69950979f7d257dff9f98bccb9f7a062f0 not found: ID does not exist"
Oct 30 21:51:09.273854 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.273820    5817 scope.go:122] "RemoveContainer" containerID="ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8"
Oct 30 21:51:09.283487 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.283456    5817 scope.go:122] "RemoveContainer" containerID="ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8"
Oct 30 21:51:09.283744 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:51:09.283725    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8\": container with ID starting with ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8 not found: ID does not exist" containerID="ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8"
Oct 30 21:51:09.283789 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.283752    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8"} err="failed to get container status \"ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8\": rpc error: code = NotFound desc = could not find container \"ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8\": container with ID starting with ccb7226611bd009de6db631e7ae15bdc71d7f6e4ba72a6dd72b809a665c395e8 not found: ID does not exist"
Oct 30 21:51:09.420138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.420111    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"]
Oct 30 21:51:09.467358 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.467322    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fbdbd87a-59f677595-wh8d2"]
Oct 30 21:51:09.529467 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.529327    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"]
Oct 30 21:51:09.536706 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:09.536680    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fbdbd87a-b8bf6db68-sz4d7"]
Oct 30 21:51:10.852645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:10.852613    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80ee7ba8-9f02-4ac1-af48-02e29f24f82d" path="/var/lib/kubelet/pods/80ee7ba8-9f02-4ac1-af48-02e29f24f82d/volumes"
Oct 30 21:51:10.853035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:10.853019    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4ab6413-5433-495d-b60c-bfb93a7b58df" path="/var/lib/kubelet/pods/a4ab6413-5433-495d-b60c-bfb93a7b58df/volumes"
Oct 30 21:51:15.434451 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.431371    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"]
Oct 30 21:51:15.440259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.438334    5817 cpu_manager.go:414] "containerMap: removing container" podUID="80ee7ba8-9f02-4ac1-af48-02e29f24f82d" containerName="payload-pre-processing"
Oct 30 21:51:15.440259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.438390    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80ee7ba8-9f02-4ac1-af48-02e29f24f82d" containerName="payload-pre-processing"
Oct 30 21:51:15.440259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.438458    5817 cpu_manager.go:414] "containerMap: removing container" podUID="a4ab6413-5433-495d-b60c-bfb93a7b58df" containerName="payload-processing"
Oct 30 21:51:15.440259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.438466    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4ab6413-5433-495d-b60c-bfb93a7b58df" containerName="payload-processing"
Oct 30 21:51:15.440259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.438928    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="80ee7ba8-9f02-4ac1-af48-02e29f24f82d" containerName="payload-pre-processing"
Oct 30 21:51:15.440259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.438964    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4ab6413-5433-495d-b60c-bfb93a7b58df" containerName="payload-processing"
Oct 30 21:51:15.446467 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.446422    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.492881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.492836    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"]
Oct 30 21:51:15.533225 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.533195    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-40bb78-openshift-default-dockercfg-p7ljt\""
Oct 30 21:51:15.549166 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549130    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-credential-socket\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549166 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549173    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-istio-token\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549457 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549330    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-envoy\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549457 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549393    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-certs\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549575 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549529    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-socket\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549629 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549578    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9faf45e5-f84c-483c-8540-01dc105b932b-istio-podinfo\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549642    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xdq2\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-kube-api-access-6xdq2\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549734 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549705    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9faf45e5-f84c-483c-8540-01dc105b932b-istiod-ca-cert\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.549812 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.549794    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-data\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651500 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651467    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-socket\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651522    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9faf45e5-f84c-483c-8540-01dc105b932b-istio-podinfo\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651583    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xdq2\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-kube-api-access-6xdq2\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651638    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9faf45e5-f84c-483c-8540-01dc105b932b-istiod-ca-cert\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651827 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651716    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-data\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651838    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-credential-socket\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651884 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651864    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-istio-token\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.651990 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.651908    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-socket\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.652054 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.652003    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-envoy\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.652111 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.652052    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-certs\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.652168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.652149    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-credential-socket\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.652386 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.652332    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-data\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.652477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.652454    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-certs\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.652668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.652646    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9faf45e5-f84c-483c-8540-01dc105b932b-istiod-ca-cert\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.654206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.654181    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-envoy\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.654296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.654233    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9faf45e5-f84c-483c-8540-01dc105b932b-istio-podinfo\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.709957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.709929    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-istio-token\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.710147 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.710124    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xdq2\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-kube-api-access-6xdq2\") pod \"e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") " pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:15.765388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:15.765311    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:16.215969 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:51:16.205591    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9faf45e5_f84c_483c_8540_01dc105b932b.slice/crio-17b63f4db72795d8e3dd2db72a56e269f1a7304829577255ab5f3152ffbd9665 WatchSource:0}: Error finding container 17b63f4db72795d8e3dd2db72a56e269f1a7304829577255ab5f3152ffbd9665: Status 404 returned error can't find the container with id 17b63f4db72795d8e3dd2db72a56e269f1a7304829577255ab5f3152ffbd9665
Oct 30 21:51:16.215969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.208494    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"]
Oct 30 21:51:16.215969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.209564    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:16.215969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.209869    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:16.215969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.209936    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:16.326946 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.326909    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" event={"ID":"9faf45e5-f84c-483c-8540-01dc105b932b","Type":"ContainerStarted","Data":"206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74"}
Oct 30 21:51:16.327157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.326977    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" event={"ID":"9faf45e5-f84c-483c-8540-01dc105b932b","Type":"ContainerStarted","Data":"17b63f4db72795d8e3dd2db72a56e269f1a7304829577255ab5f3152ffbd9665"}
Oct 30 21:51:16.341319 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:51:16.341285    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:51:16.444093 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.444032    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podStartSLOduration=1.444016978 podStartE2EDuration="1.444016978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:16.44219264 +0000 UTC m=+2211.729915208" watchObservedRunningTime="2026-10-30 21:51:16.444016978 +0000 UTC m=+2211.731739524"
Oct 30 21:51:16.765443 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:16.765409    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:17.771399 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:17.771367    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:18.341834 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:18.341803    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:18.342645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:18.342623    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:51:24.113371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.112630    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"]
Oct 30 21:51:24.127373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.118816    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.173405 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.166217    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"]
Oct 30 21:51:24.177199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.177149    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fa220f-5ded-45b4-94b5-400893ffdac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.177500 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.177455    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wknjc\" (UniqueName: \"kubernetes.io/projected/f7fa220f-5ded-45b4-94b5-400893ffdac5-kube-api-access-wknjc\") pod \"payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.186817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.186768    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-40bb78-dockercfg-497fz\""
Oct 30 21:51:24.236273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.236238    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-40bb78\""
Oct 30 21:51:24.279634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.279597    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fa220f-5ded-45b4-94b5-400893ffdac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.279825 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.279646    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wknjc\" (UniqueName: \"kubernetes.io/projected/f7fa220f-5ded-45b4-94b5-400893ffdac5-kube-api-access-wknjc\") pod \"payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.280646 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.280620    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fa220f-5ded-45b4-94b5-400893ffdac5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.338435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.338395    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wknjc\" (UniqueName: \"kubernetes.io/projected/f7fa220f-5ded-45b4-94b5-400893ffdac5-kube-api-access-wknjc\") pod \"payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.536603 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.536576    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:24.943393 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:24.943334    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"]
Oct 30 21:51:25.540834 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:25.540799    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r" event={"ID":"f7fa220f-5ded-45b4-94b5-400893ffdac5","Type":"ContainerStarted","Data":"9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54"}
Oct 30 21:51:25.541308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:25.541290    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:25.541450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:25.541434    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r" event={"ID":"f7fa220f-5ded-45b4-94b5-400893ffdac5","Type":"ContainerStarted","Data":"8ac333ed2a016d5c15674ce75217bd2d64b0a2d66dd0ea818ff0c7bf9e129f6c"}
Oct 30 21:51:25.672084 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:25.672034    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r" podStartSLOduration=1.672023184 podStartE2EDuration="1.672023184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:25.668939072 +0000 UTC m=+2220.956661618" watchObservedRunningTime="2026-10-30 21:51:25.672023184 +0000 UTC m=+2220.959745731"
Oct 30 21:51:26.860930 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:26.860890    5817 scope.go:122] "RemoveContainer" containerID="88eaf3b50dba9148acf4deac3f68927c663beec1d0fbbf90cc10d74bbd81946a"
Oct 30 21:51:30.596593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:30.596560    5817 generic.go:356] "Generic (PLEG): container finished" podID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerID="5ca32526f9bb2988d78ae56c0e7cd218e8caf1cdc2545a58cc91e8e12b4c1d99" exitCode=0
Oct 30 21:51:30.596939 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:30.596623    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" event={"ID":"fd7217ba-460e-45c6-804b-70ebd50c3d38","Type":"ContainerDied","Data":"5ca32526f9bb2988d78ae56c0e7cd218e8caf1cdc2545a58cc91e8e12b4c1d99"}
Oct 30 21:51:30.597102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:30.597084    5817 scope.go:122] "RemoveContainer" containerID="5ca32526f9bb2988d78ae56c0e7cd218e8caf1cdc2545a58cc91e8e12b4c1d99"
Oct 30 21:51:30.598970 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:30.598937    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:30.599084 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:30.599017    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:30.599084 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:30.599054    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:31.620652 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:31.620613    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" event={"ID":"fd7217ba-460e-45c6-804b-70ebd50c3d38","Type":"ContainerStarted","Data":"7769fe88a892751ec9bb095c4097637ee58968ae12b63d13a81dc0f6d756651e"}
Oct 30 21:51:32.243505 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:32.243469    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:51:32.244499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:32.244472    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:51:32.244627 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:32.244529    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 21:51:33.243017 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:33.242984    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:51:33.243455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:33.243036    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 21:51:33.463114 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:33.463079    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:51:33.568382 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:33.568228    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"]
Oct 30 21:51:33.569486 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:33.569150    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8" podUID="2c353ed2-61e8-46fd-a388-8d337921f529" containerName="maas-api" containerID="cri-o://e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b" gracePeriod=30
Oct 30 21:51:34.167375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.164524    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:51:34.179368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.176968    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c353ed2-61e8-46fd-a388-8d337921f529-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c353ed2-61e8-46fd-a388-8d337921f529-openshift-service-ca\") pod \"2c353ed2-61e8-46fd-a388-8d337921f529\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") "
Oct 30 21:51:34.179368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.177434    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c353ed2-61e8-46fd-a388-8d337921f529-openshift-service-ca" pod "2c353ed2-61e8-46fd-a388-8d337921f529" (UID: "2c353ed2-61e8-46fd-a388-8d337921f529"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:34.179368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.177820    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls\") pod \"2c353ed2-61e8-46fd-a388-8d337921f529\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") "
Oct 30 21:51:34.179368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.178240    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-metrics-certs\") pod \"2c353ed2-61e8-46fd-a388-8d337921f529\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") "
Oct 30 21:51:34.179368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.178669    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c353ed2-61e8-46fd-a388-8d337921f529-kube-api-access-j5g2v\" (UniqueName: \"kubernetes.io/projected/2c353ed2-61e8-46fd-a388-8d337921f529-kube-api-access-j5g2v\") pod \"2c353ed2-61e8-46fd-a388-8d337921f529\" (UID: \"2c353ed2-61e8-46fd-a388-8d337921f529\") "
Oct 30 21:51:34.179770 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.179565    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c353ed2-61e8-46fd-a388-8d337921f529-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.184610 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.184566    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls" pod "2c353ed2-61e8-46fd-a388-8d337921f529" (UID: "2c353ed2-61e8-46fd-a388-8d337921f529"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:34.186533 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.185643    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c353ed2-61e8-46fd-a388-8d337921f529-kube-api-access-j5g2v" pod "2c353ed2-61e8-46fd-a388-8d337921f529" (UID: "2c353ed2-61e8-46fd-a388-8d337921f529"). InnerVolumeSpecName "kube-api-access-j5g2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:34.214419 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.201514    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-metrics-certs" pod "2c353ed2-61e8-46fd-a388-8d337921f529" (UID: "2c353ed2-61e8-46fd-a388-8d337921f529"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:51:34.243668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.243640    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:51:34.243980 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.243689    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 21:51:34.281180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.281152    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.281180 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.281181    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c353ed2-61e8-46fd-a388-8d337921f529-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.281339 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.281196    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5g2v\" (UniqueName: \"kubernetes.io/projected/2c353ed2-61e8-46fd-a388-8d337921f529-kube-api-access-j5g2v\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:34.839803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.839764    5817 generic.go:356] "Generic (PLEG): container finished" podID="2c353ed2-61e8-46fd-a388-8d337921f529" containerID="e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b" exitCode=0
Oct 30 21:51:34.844978 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.844949    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"
Oct 30 21:51:34.848866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.848841    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8" event={"ID":"2c353ed2-61e8-46fd-a388-8d337921f529","Type":"ContainerDied","Data":"e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b"}
Oct 30 21:51:34.849025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.848892    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8" event={"ID":"2c353ed2-61e8-46fd-a388-8d337921f529","Type":"ContainerDied","Data":"4ff6249eab69833fe21617655949fd7aef004746e503d004e182404504633b22"}
Oct 30 21:51:34.849025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.848919    5817 scope.go:122] "RemoveContainer" containerID="e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b"
Oct 30 21:51:34.865895 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.865856    5817 scope.go:122] "RemoveContainer" containerID="e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b"
Oct 30 21:51:34.866451 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:51:34.866425    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b\": container with ID starting with e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b not found: ID does not exist" containerID="e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b"
Oct 30 21:51:34.866549 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:34.866456    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b"} err="failed to get container status \"e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b\": rpc error: code = NotFound desc = could not find container \"e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b\": container with ID starting with e943e5e416e1d85c00ee455be94cd71c1c361aeec4288c66097736c6c63c618b not found: ID does not exist"
Oct 30 21:51:35.024714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:35.024665    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"]
Oct 30 21:51:35.061606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:35.061570    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fbdbd87a-7cb65c56d4-4nhc8"]
Oct 30 21:51:35.242869 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:35.242831    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:51:35.243093 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:35.242897    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 21:51:35.858800 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:35.858745    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"]
Oct 30 21:51:35.859571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:35.859540    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy" containerID="cri-o://395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d" gracePeriod=30
Oct 30 21:51:36.243064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:36.243025    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 30 21:51:36.243266 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:36.243075    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 30 21:51:36.846716 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:36.846682    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c353ed2-61e8-46fd-a388-8d337921f529" path="/var/lib/kubelet/pods/2c353ed2-61e8-46fd-a388-8d337921f529/volumes"
Oct 30 21:51:37.277839 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:37.277340    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:51:37.289577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:37.289547    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 21:51:37.574407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:37.574278    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 21:51:38.133492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.133422    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"]
Oct 30 21:51:38.134273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.134255    5817 cpu_manager.go:414] "containerMap: removing container" podUID="2c353ed2-61e8-46fd-a388-8d337921f529" containerName="maas-api"
Oct 30 21:51:38.134321 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.134276    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c353ed2-61e8-46fd-a388-8d337921f529" containerName="maas-api"
Oct 30 21:51:38.134420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.134410    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c353ed2-61e8-46fd-a388-8d337921f529" containerName="maas-api"
Oct 30 21:51:38.137713 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.137694    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.186938 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.186890    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-27114d-openshift-default-dockercfg-qd9rv\""
Oct 30 21:51:38.209864 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.209833    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"]
Oct 30 21:51:38.215081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215059    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-certs\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215153 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215098    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-socket\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215360 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215318    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-token\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215422    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-data\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215521    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-podinfo\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215565    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbjsc\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-kube-api-access-nbjsc\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215684 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215670    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-credential-socket\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215752 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215730    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-envoy\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.215814 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.215769    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istiod-ca-cert\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.317033 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.316984    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-socket\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.317577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317138    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-token\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.317577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317164    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-data\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.317577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317290    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-podinfo\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.317577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317334    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbjsc\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-kube-api-access-nbjsc\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.317577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317465    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-credential-socket\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.317577 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317507    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-envoy\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.318046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317844    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-credential-socket\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.318046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317889    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-socket\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.318162 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.317543    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istiod-ca-cert\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.318277 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.318251    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-certs\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.318472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.318027    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-data\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.318619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.318585    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istiod-ca-cert\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.318684 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.318662    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-certs\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.320643 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.320621    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-envoy\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.321053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.321029    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-podinfo\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.375472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.375429    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-token\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.375635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.375551    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbjsc\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-kube-api-access-nbjsc\") pod \"e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") " pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.450703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.450670    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:38.880246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.880217    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"]
Oct 30 21:51:38.880428 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:51:38.880407    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19cac82a_c4ed_4d20_8ed2_07d1bd6e3c5d.slice/crio-491bb01e65f2e8f6c88c503e3843c7f69193654ee912a96326c929390a59bf3d WatchSource:0}: Error finding container 491bb01e65f2e8f6c88c503e3843c7f69193654ee912a96326c929390a59bf3d: Status 404 returned error can't find the container with id 491bb01e65f2e8f6c88c503e3843c7f69193654ee912a96326c929390a59bf3d
Oct 30 21:51:38.882856 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.882820    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:38.882975 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.882896    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:38.882975 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.882940    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:38.901620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:38.901596    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" event={"ID":"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d","Type":"ContainerStarted","Data":"491bb01e65f2e8f6c88c503e3843c7f69193654ee912a96326c929390a59bf3d"}
Oct 30 21:51:39.910398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:39.910286    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" event={"ID":"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d","Type":"ContainerStarted","Data":"3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae"}
Oct 30 21:51:40.122862 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:40.122179    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podStartSLOduration=2.122161051 podStartE2EDuration="2.122161051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:40.072791014 +0000 UTC m=+2235.360513560" watchObservedRunningTime="2026-10-30 21:51:40.122161051 +0000 UTC m=+2235.409883600"
Oct 30 21:51:40.451259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:40.451226    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:40.458840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:40.458807    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:40.917766 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:40.917744    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:40.918529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:40.918510    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:51:41.224008 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.223980    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:51:41.370153 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370120    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bace0d0d-712e-4138-88da-98c90e39413a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bace0d0d-712e-4138-88da-98c90e39413a-istio-podinfo\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.370308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370198    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-istio-token\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-istio-token\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.370308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370277    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-certs\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.370411 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370374    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-envoy\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.370503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370483    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-socket\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.370904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370622    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-kube-api-access-xtwsp\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-kube-api-access-xtwsp\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.370904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370642    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-certs" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.370904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370662    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-credential-socket\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.370904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.370799    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bace0d0d-712e-4138-88da-98c90e39413a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bace0d0d-712e-4138-88da-98c90e39413a-istiod-ca-cert\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.373227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373191    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bace0d0d-712e-4138-88da-98c90e39413a-istiod-ca-cert" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:51:41.373370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373245    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-socket" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.373370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373324    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-envoy" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.373499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373472    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-istio-token" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:41.373829 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373547    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bace0d0d-712e-4138-88da-98c90e39413a-istio-podinfo" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:51:41.373951 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373850    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-kube-api-access-xtwsp" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "kube-api-access-xtwsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:51:41.374055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373956    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-credential-socket" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.374144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.374097    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-data" pod "bace0d0d-712e-4138-88da-98c90e39413a" (UID: "bace0d0d-712e-4138-88da-98c90e39413a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:51:41.374220 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.373900    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-data\") pod \"bace0d0d-712e-4138-88da-98c90e39413a\" (UID: \"bace0d0d-712e-4138-88da-98c90e39413a\") "
Oct 30 21:51:41.375868 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.375845    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bace0d0d-712e-4138-88da-98c90e39413a-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376007 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.375993    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.376089    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bace0d0d-712e-4138-88da-98c90e39413a-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376200 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.376186    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376281 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.376271    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.376339    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.376459    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.376568    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtwsp\" (UniqueName: \"kubernetes.io/projected/bace0d0d-712e-4138-88da-98c90e39413a-kube-api-access-xtwsp\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.376677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.376664    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bace0d0d-712e-4138-88da-98c90e39413a-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:51:41.926409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.926378    5817 generic.go:356] "Generic (PLEG): container finished" podID="bace0d0d-712e-4138-88da-98c90e39413a" containerID="395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d" exitCode=0
Oct 30 21:51:41.926853 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.926424    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"
Oct 30 21:51:41.926853 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.926462    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" event={"ID":"bace0d0d-712e-4138-88da-98c90e39413a","Type":"ContainerDied","Data":"395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d"}
Oct 30 21:51:41.926853 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.926515    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4" event={"ID":"bace0d0d-712e-4138-88da-98c90e39413a","Type":"ContainerDied","Data":"dcd76149b46ba7f1d10eb9c3e86a34a9f9e98a38b2ce929fc23dde0d227ef587"}
Oct 30 21:51:41.926853 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.926536    5817 scope.go:122] "RemoveContainer" containerID="395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d"
Oct 30 21:51:41.938986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.938969    5817 scope.go:122] "RemoveContainer" containerID="4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02"
Oct 30 21:51:41.948514 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.948495    5817 scope.go:122] "RemoveContainer" containerID="395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d"
Oct 30 21:51:41.948765 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:51:41.948748    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d\": container with ID starting with 395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d not found: ID does not exist" containerID="395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d"
Oct 30 21:51:41.948817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.948773    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d"} err="failed to get container status \"395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d\": rpc error: code = NotFound desc = could not find container \"395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d\": container with ID starting with 395324a739bfcb8ce64f23db2e261394413a123d37aa1d27a514186803120e0d not found: ID does not exist"
Oct 30 21:51:41.948817 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.948789    5817 scope.go:122] "RemoveContainer" containerID="4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02"
Oct 30 21:51:41.949005 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:51:41.948983    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02\": container with ID starting with 4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02 not found: ID does not exist" containerID="4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02"
Oct 30 21:51:41.949049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:41.949013    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02"} err="failed to get container status \"4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02\": rpc error: code = NotFound desc = could not find container \"4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02\": container with ID starting with 4c4ec2e3a304773893c513aa0232195bb69538938fbd24c9d8089067f0470b02 not found: ID does not exist"
Oct 30 21:51:42.088404 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:42.088240    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"]
Oct 30 21:51:42.093166 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:42.093138    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fbdbd87a-gw-openshift-default-568f47bf57-c69p4"]
Oct 30 21:51:42.845203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:42.845169    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bace0d0d-712e-4138-88da-98c90e39413a" path="/var/lib/kubelet/pods/bace0d0d-712e-4138-88da-98c90e39413a/volumes"
Oct 30 21:51:46.339661 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:51:46.339606    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:51:53.020666 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.020624    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"]
Oct 30 21:51:53.022525 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.022497    5817 cpu_manager.go:414] "containerMap: removing container" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy"
Oct 30 21:51:53.022525 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.022523    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy"
Oct 30 21:51:53.022718 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.022590    5817 cpu_manager.go:414] "containerMap: removing container" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy"
Oct 30 21:51:53.022718 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.022598    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy"
Oct 30 21:51:53.022985 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.022966    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy"
Oct 30 21:51:53.023046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.023000    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="bace0d0d-712e-4138-88da-98c90e39413a" containerName="istio-proxy"
Oct 30 21:51:53.027637 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.027611    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.100615 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.100571    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-data\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.100882 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.100860    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-envoy\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.101041 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.101021    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-podinfo\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.101275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.101256    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-certs\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.101450 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.101433    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istiod-ca-cert\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.101608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.101590    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7p8p\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-kube-api-access-b7p8p\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.101733 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.101718    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-token\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.101903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.101888    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-socket\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.102077 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.102063    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-credential-socket\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.162458 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.161269    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-737a64-openshift-default-dockercfg-lcrm5\""
Oct 30 21:51:53.173729 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.172848    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"]
Oct 30 21:51:53.204466 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.204394    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-certs\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.205241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.205216    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istiod-ca-cert\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.205464 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.205445    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7p8p\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-kube-api-access-b7p8p\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.205616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.205598    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-token\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.205794 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.205778    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-socket\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.206015 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.205999    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-credential-socket\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.206227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.206209    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-data\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.206373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.206340    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-envoy\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.208073 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.208049    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-podinfo\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.208976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.205121    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-certs\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.209844 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.209822    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istiod-ca-cert\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.210509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.210485    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-credential-socket\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.211722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.211701    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-socket\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.212157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.212135    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-data\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.212281 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.212134    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-podinfo\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.238384 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.226938    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-envoy\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.272140 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.272046    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7p8p\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-kube-api-access-b7p8p\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.273569 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.273540    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-token\") pod \"e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") " pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.464183 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.464146    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:53.932495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.932415    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"]
Oct 30 21:51:53.961332 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:51:53.961303    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eba6ac8_c7e3_435f_a1d6_e6877cb015bc.slice/crio-780b055afd6a9766bf268d6ee5deaa5784179c4adb7d8a5e74834abd2681f9e5 WatchSource:0}: Error finding container 780b055afd6a9766bf268d6ee5deaa5784179c4adb7d8a5e74834abd2681f9e5: Status 404 returned error can't find the container with id 780b055afd6a9766bf268d6ee5deaa5784179c4adb7d8a5e74834abd2681f9e5
Oct 30 21:51:53.964057 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.964023    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:53.964147 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.964102    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:53.964185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:53.964147    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:51:54.183874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:54.183792    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" event={"ID":"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc","Type":"ContainerStarted","Data":"183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5"}
Oct 30 21:51:54.183874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:54.183850    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" event={"ID":"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc","Type":"ContainerStarted","Data":"780b055afd6a9766bf268d6ee5deaa5784179c4adb7d8a5e74834abd2681f9e5"}
Oct 30 21:51:54.310661 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:54.310598    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podStartSLOduration=2.310579845 podStartE2EDuration="2.310579845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:54.305481634 +0000 UTC m=+2249.593204182" watchObservedRunningTime="2026-10-30 21:51:54.310579845 +0000 UTC m=+2249.598302392"
Oct 30 21:51:54.465650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:54.465598    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:55.472568 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:55.472529    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:56.254477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:56.254446    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:51:56.265138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:51:56.265095    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:52:16.338147 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:52:16.338069    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:52:21.497069 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.497032    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"]
Oct 30 21:52:21.497555 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.497314    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy" containerID="cri-o://b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2" gracePeriod=30
Oct 30 21:52:21.543843 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.543811    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"]
Oct 30 21:52:21.544010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.543991    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" containerID="cri-o://73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb" gracePeriod=30
Oct 30 21:52:21.547792 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.547765    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"]
Oct 30 21:52:21.547989 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.547965    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" containerID="cri-o://465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508" gracePeriod=30
Oct 30 21:52:21.592020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.591991    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"]
Oct 30 21:52:21.592246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:21.592206    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" containerID="cri-o://bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30" gracePeriod=30
Oct 30 21:52:25.966773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:25.966738    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:52:25.967150 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:25.966797    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:52:26.646719 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:26.646678    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 21:52:26.646873 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:26.646747    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 21:52:26.911252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:26.903033    5817 scope.go:122] "RemoveContainer" containerID="d057e54e028dd9f4aaae8fe29fa123bae45a852ee2571386fd7b0d5600222cc0"
Oct 30 21:52:26.942236 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:26.941815    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:52:26.970744 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:26.970715    5817 scope.go:122] "RemoveContainer" containerID="7bb86c03431c663a0c69b12e7c4231250d7959738ac5a2b863be2177a8b736e1"
Oct 30 21:52:27.008992 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.008916    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:52:27.014901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.014878    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:52:27.027564 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.027540    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:52:27.062788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.062766    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-certs\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.062892 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.062854    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-socket\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.062935 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.062911    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-envoy\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.062969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.062953    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-credential-socket\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.063040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.062990    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-certs" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.063076 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063067    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-token\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-token\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.063161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063132    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-socket" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.063161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063145    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istiod-ca-cert\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.063275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063213    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-kube-api-access-fwpq8\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-kube-api-access-fwpq8\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.063275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063248    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-podinfo\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.063434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063298    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-credential-socket" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.063434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063333    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-data\") pod \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\" (UID: \"2c8f0730-9523-4719-a5d9-f3aee98bb8b5\") "
Oct 30 21:52:27.063644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063616    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-data" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.063729 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.063637    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istiod-ca-cert" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:27.064261 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.064233    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.064386 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.064264    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.064386 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.064279    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.064386 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.064293    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.064386 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.064309    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.065228 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.065199    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-envoy" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.065325 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.065306    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-token" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.065459 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.065385    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-kube-api-access-fwpq8" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "kube-api-access-fwpq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.065510 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.065493    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-podinfo" pod "2c8f0730-9523-4719-a5d9-f3aee98bb8b5" (UID: "2c8f0730-9523-4719-a5d9-f3aee98bb8b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:27.165284 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165201    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-credential-socket\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.165284 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165253    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-data\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.165522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165291    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-envoy\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.165522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165318    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-data\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.165522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165411    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f507f65-d132-4b31-acfa-0cc08bd2a732-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f507f65-d132-4b31-acfa-0cc08bd2a732-istiod-ca-cert\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.165522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165466    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-credential-socket\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.165522 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165499    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-token\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-token\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.165797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165500    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-credential-socket" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.165797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165592    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-data" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.165797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165668    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-podinfo\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.165797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165771    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-socket\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.165797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165669    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-data" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.166055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165829    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-data\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.166055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165862    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-envoy\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.166055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165861    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f507f65-d132-4b31-acfa-0cc08bd2a732-istiod-ca-cert" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:27.166055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165770    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-credential-socket" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.166055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165962    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d84241f-3219-4f7c-9af6-084c34f8b986-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d84241f-3219-4f7c-9af6-084c34f8b986-istiod-ca-cert\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.166055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165984    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-socket" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.166055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.165993    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-certs\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166089    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-data" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166143    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-kube-api-access-jk8j2\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-kube-api-access-jk8j2\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166176    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-credential-socket\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166253    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-podinfo\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166298    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-token\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-token\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166337    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-certs\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166395    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-kube-api-access-dcc2z\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-kube-api-access-dcc2z\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166425    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-token\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-token\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166486    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-podinfo\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166535    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-certs\") pod \"0d84241f-3219-4f7c-9af6-084c34f8b986\" (UID: \"0d84241f-3219-4f7c-9af6-084c34f8b986\") "
Oct 30 21:52:27.166616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166567    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-socket\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.167039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166655    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55cf8731-65de-4bac-abdf-9c7608ed42bf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55cf8731-65de-4bac-abdf-9c7608ed42bf-istiod-ca-cert\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.167039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166733    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-kube-api-access-dmxm7\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-kube-api-access-dmxm7\") pod \"55cf8731-65de-4bac-abdf-9c7608ed42bf\" (UID: \"55cf8731-65de-4bac-abdf-9c7608ed42bf\") "
Oct 30 21:52:27.167039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166771    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-envoy\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.167039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166850    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-socket\") pod \"1f507f65-d132-4b31-acfa-0cc08bd2a732\" (UID: \"1f507f65-d132-4b31-acfa-0cc08bd2a732\") "
Oct 30 21:52:27.167741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167721    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167747    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167760    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167773    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f507f65-d132-4b31-acfa-0cc08bd2a732-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167785    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167798    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167812    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167825    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167838    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167852    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwpq8\" (UniqueName: \"kubernetes.io/projected/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-kube-api-access-fwpq8\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.167947 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.167866    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c8f0730-9523-4719-a5d9-f3aee98bb8b5-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.169025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166396    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-certs" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169025 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.166413    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d84241f-3219-4f7c-9af6-084c34f8b986-istiod-ca-cert" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168122    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-socket" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168311    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-certs" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168568    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-socket" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168597    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-envoy" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168782    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-podinfo" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168806    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-kube-api-access-jk8j2" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "kube-api-access-jk8j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168857    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55cf8731-65de-4bac-abdf-9c7608ed42bf-istiod-ca-cert" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168894    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-token" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.169199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.168996    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-credential-socket" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169747 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.169429    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-certs" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169747 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.169526    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-envoy" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.169943 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.169915    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-podinfo" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:27.170435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.170410    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-podinfo" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:27.170733 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.170715    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-kube-api-access-dcc2z" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "kube-api-access-dcc2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.171024 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.170999    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-token" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.171133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.171116    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-kube-api-access-dmxm7" pod "55cf8731-65de-4bac-abdf-9c7608ed42bf" (UID: "55cf8731-65de-4bac-abdf-9c7608ed42bf"). InnerVolumeSpecName "kube-api-access-dmxm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.171444 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.171426    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-token" pod "0d84241f-3219-4f7c-9af6-084c34f8b986" (UID: "0d84241f-3219-4f7c-9af6-084c34f8b986"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:27.171502 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.171482    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-envoy" pod "1f507f65-d132-4b31-acfa-0cc08bd2a732" (UID: "1f507f65-d132-4b31-acfa-0cc08bd2a732"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:27.270049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.269950    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.269997    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270017    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270034    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270049    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d84241f-3219-4f7c-9af6-084c34f8b986-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270065    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270049 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270080    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk8j2\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-kube-api-access-jk8j2\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270095    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270109    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270122    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270136    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270150    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcc2z\" (UniqueName: \"kubernetes.io/projected/1f507f65-d132-4b31-acfa-0cc08bd2a732-kube-api-access-dcc2z\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270162    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d84241f-3219-4f7c-9af6-084c34f8b986-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270175    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55cf8731-65de-4bac-abdf-9c7608ed42bf-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270188    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d84241f-3219-4f7c-9af6-084c34f8b986-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270200    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55cf8731-65de-4bac-abdf-9c7608ed42bf-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270213    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55cf8731-65de-4bac-abdf-9c7608ed42bf-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270227    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmxm7\" (UniqueName: \"kubernetes.io/projected/55cf8731-65de-4bac-abdf-9c7608ed42bf-kube-api-access-dmxm7\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270240    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.270513 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.270254    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f507f65-d132-4b31-acfa-0cc08bd2a732-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:27.586292 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.586260    5817 generic.go:356] "Generic (PLEG): container finished" podID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerID="465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508" exitCode=0
Oct 30 21:52:27.586532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.586305    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"
Oct 30 21:52:27.586532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.586336    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" event={"ID":"1f507f65-d132-4b31-acfa-0cc08bd2a732","Type":"ContainerDied","Data":"465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508"}
Oct 30 21:52:27.586532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.586419    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd" event={"ID":"1f507f65-d132-4b31-acfa-0cc08bd2a732","Type":"ContainerDied","Data":"dafc4850af2bf169dffee1063dce1ef971b5be7ae16bfb607b7211ef034d7de2"}
Oct 30 21:52:27.586532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.586445    5817 scope.go:122] "RemoveContainer" containerID="465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508"
Oct 30 21:52:27.587850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.587825    5817 generic.go:356] "Generic (PLEG): container finished" podID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerID="bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30" exitCode=0
Oct 30 21:52:27.587967 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.587870    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"
Oct 30 21:52:27.587967 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.587912    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" event={"ID":"55cf8731-65de-4bac-abdf-9c7608ed42bf","Type":"ContainerDied","Data":"bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30"}
Oct 30 21:52:27.587967 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.587963    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" event={"ID":"55cf8731-65de-4bac-abdf-9c7608ed42bf","Type":"ContainerDied","Data":"204a75196eaad9fba04c78ef0b4bedc9a0cb51730e774bcdc97aac4a6a9ea233"}
Oct 30 21:52:27.589774 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.589745    5817 generic.go:356] "Generic (PLEG): container finished" podID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerID="b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2" exitCode=0
Oct 30 21:52:27.589901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.589831    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" event={"ID":"2c8f0730-9523-4719-a5d9-f3aee98bb8b5","Type":"ContainerDied","Data":"b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2"}
Oct 30 21:52:27.589901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.589873    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j" event={"ID":"2c8f0730-9523-4719-a5d9-f3aee98bb8b5","Type":"ContainerDied","Data":"20068f32af37b3d1cf5dbf6f04c969c53bba956decff1eeabc7da79f3342aa05"}
Oct 30 21:52:27.590017 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.589953    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"
Oct 30 21:52:27.596086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.596057    5817 generic.go:356] "Generic (PLEG): container finished" podID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerID="73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb" exitCode=0
Oct 30 21:52:27.596086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.596111    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" event={"ID":"0d84241f-3219-4f7c-9af6-084c34f8b986","Type":"ContainerDied","Data":"73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb"}
Oct 30 21:52:27.596413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.596152    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb" event={"ID":"0d84241f-3219-4f7c-9af6-084c34f8b986","Type":"ContainerDied","Data":"0316b56d9e8177cee54ff613423e20180e3c853c6e7db57899ac6b4e327ccfd4"}
Oct 30 21:52:27.596413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.596323    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"
Oct 30 21:52:27.601694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.601652    5817 scope.go:122] "RemoveContainer" containerID="465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508"
Oct 30 21:52:27.602238 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:52:27.602216    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508\": container with ID starting with 465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508 not found: ID does not exist" containerID="465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508"
Oct 30 21:52:27.602317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.602244    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508"} err="failed to get container status \"465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508\": rpc error: code = NotFound desc = could not find container \"465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508\": container with ID starting with 465c7621155f5fdab568e8348061eb3221682f941549a8e35fd668854a9d6508 not found: ID does not exist"
Oct 30 21:52:27.602317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.602263    5817 scope.go:122] "RemoveContainer" containerID="bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30"
Oct 30 21:52:27.617766 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.617743    5817 scope.go:122] "RemoveContainer" containerID="bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30"
Oct 30 21:52:27.618087 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:52:27.618067    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30\": container with ID starting with bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30 not found: ID does not exist" containerID="bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30"
Oct 30 21:52:27.618131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.618096    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30"} err="failed to get container status \"bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30\": rpc error: code = NotFound desc = could not find container \"bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30\": container with ID starting with bc3458bea78f34be9b416d7a8e87bc6cdb89971a9ef934cb57f25ddbb4bf3b30 not found: ID does not exist"
Oct 30 21:52:27.618131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.618117    5817 scope.go:122] "RemoveContainer" containerID="b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2"
Oct 30 21:52:27.628560 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.628540    5817 scope.go:122] "RemoveContainer" containerID="b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2"
Oct 30 21:52:27.628826 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:52:27.628806    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2\": container with ID starting with b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2 not found: ID does not exist" containerID="b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2"
Oct 30 21:52:27.628886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.628834    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2"} err="failed to get container status \"b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2\": rpc error: code = NotFound desc = could not find container \"b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2\": container with ID starting with b4a2bb01993426224cbccd8d4f96cc7ccf536043a621e9894f2c32f13cb35ac2 not found: ID does not exist"
Oct 30 21:52:27.628886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.628851    5817 scope.go:122] "RemoveContainer" containerID="73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb"
Oct 30 21:52:27.639006 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.638984    5817 scope.go:122] "RemoveContainer" containerID="73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb"
Oct 30 21:52:27.639293 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:52:27.639270    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb\": container with ID starting with 73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb not found: ID does not exist" containerID="73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb"
Oct 30 21:52:27.639432 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.639303    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb"} err="failed to get container status \"73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb\": rpc error: code = NotFound desc = could not find container \"73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb\": container with ID starting with 73a6fd1e9f3c1f08473a5ecebd42b49fd59f4514399b50b62910858eab118adb not found: ID does not exist"
Oct 30 21:52:27.747402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.747361    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"]
Oct 30 21:52:27.751697 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.751669    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8"]
Oct 30 21:52:27.856329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.856243    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"]
Oct 30 21:52:27.864313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.864277    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-5mtrd"]
Oct 30 21:52:27.870133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.870103    5817 patch_prober.go:29] interesting pod/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:52:27.870251 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:27.870165    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-rfzl8" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:28.032937 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.032897    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"]
Oct 30 21:52:28.037299 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.037267    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-4ks9j"]
Oct 30 21:52:28.142931 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.142832    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"]
Oct 30 21:52:28.146729 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.146697    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-afbe5b-openshift-default-65fbcd9cf5-fk5lb"]
Oct 30 21:52:28.845088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.845050    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" path="/var/lib/kubelet/pods/0d84241f-3219-4f7c-9af6-084c34f8b986/volumes"
Oct 30 21:52:28.845846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.845825    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" path="/var/lib/kubelet/pods/1f507f65-d132-4b31-acfa-0cc08bd2a732/volumes"
Oct 30 21:52:28.846329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.846314    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" path="/var/lib/kubelet/pods/2c8f0730-9523-4719-a5d9-f3aee98bb8b5/volumes"
Oct 30 21:52:28.847104 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:28.847087    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" path="/var/lib/kubelet/pods/55cf8731-65de-4bac-abdf-9c7608ed42bf/volumes"
Oct 30 21:52:35.550816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.550781    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"]
Oct 30 21:52:35.551521 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551505    5817 cpu_manager.go:414] "containerMap: removing container" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy"
Oct 30 21:52:35.551521 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551522    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551532    5817 cpu_manager.go:414] "containerMap: removing container" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551537    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551559    5817 cpu_manager.go:414] "containerMap: removing container" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551565    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551577    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551582    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551597    5817 cpu_manager.go:414] "containerMap: removing container" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy"
Oct 30 21:52:35.551611 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551603    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy"
Oct 30 21:52:35.551840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551718    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy"
Oct 30 21:52:35.551840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551727    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d84241f-3219-4f7c-9af6-084c34f8b986" containerName="istio-proxy"
Oct 30 21:52:35.551840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551735    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c8f0730-9523-4719-a5d9-f3aee98bb8b5" containerName="istio-proxy"
Oct 30 21:52:35.551840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551748    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f507f65-d132-4b31-acfa-0cc08bd2a732" containerName="istio-proxy"
Oct 30 21:52:35.551840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.551754    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="55cf8731-65de-4bac-abdf-9c7608ed42bf" containerName="istio-proxy"
Oct 30 21:52:35.557580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.557556    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.602497 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.602469    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.602620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.602508    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.602620 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.602534    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34de55-a99d-4f02-b45e-915395b7d0b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.602740 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.602720    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2v8c\" (UniqueName: \"kubernetes.io/projected/7d34de55-a99d-4f02-b45e-915395b7d0b4-kube-api-access-b2v8c\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.602802 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.602787    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.602885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.602868    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.602975 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.602951    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.603034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.603002    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.603131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.603111    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.674255 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.674223    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"]
Oct 30 21:52:35.704294 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704257    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.704471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704438    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.704533 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704486    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.704596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704528    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34de55-a99d-4f02-b45e-915395b7d0b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.704759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704733    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-workload-certs\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.704872 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704854    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2v8c\" (UniqueName: \"kubernetes.io/projected/7d34de55-a99d-4f02-b45e-915395b7d0b4-kube-api-access-b2v8c\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.704940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704918    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-credential-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.705004 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.704946    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.705064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.705002    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.705064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.705038    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.705163 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.705075    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.705409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.705386    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34de55-a99d-4f02-b45e-915395b7d0b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.705566 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.705424    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-data\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.705682 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.705652    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-workload-socket\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.708381 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.708317    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.715644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.715618    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.764963 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.764928    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34de55-a99d-4f02-b45e-915395b7d0b4-istio-token\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.773235 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.773200    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2v8c\" (UniqueName: \"kubernetes.io/projected/7d34de55-a99d-4f02-b45e-915395b7d0b4-kube-api-access-b2v8c\") pod \"maas-default-gateway-openshift-default-67bf986f99-jsclw\" (UID: \"7d34de55-a99d-4f02-b45e-915395b7d0b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:35.872586 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:35.872507    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:36.300745 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:52:36.300709    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d34de55_a99d_4f02_b45e_915395b7d0b4.slice/crio-f433573ca3cc719e7fd8526af3cd3f3c5aea6f0ccb627552b138dc872bb0a129 WatchSource:0}: Error finding container f433573ca3cc719e7fd8526af3cd3f3c5aea6f0ccb627552b138dc872bb0a129: Status 404 returned error can't find the container with id f433573ca3cc719e7fd8526af3cd3f3c5aea6f0ccb627552b138dc872bb0a129
Oct 30 21:52:36.302094 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.302070    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"]
Oct 30 21:52:36.304018 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.303975    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:36.304133 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.304078    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:36.304197 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.304129    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:36.672904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.672725    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" event={"ID":"7d34de55-a99d-4f02-b45e-915395b7d0b4","Type":"ContainerStarted","Data":"47632495b9a96d8802f7f152e718caf2e26b12d75202ec2e37e0c6f5a2e4745a"}
Oct 30 21:52:36.672904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.672799    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" event={"ID":"7d34de55-a99d-4f02-b45e-915395b7d0b4","Type":"ContainerStarted","Data":"f433573ca3cc719e7fd8526af3cd3f3c5aea6f0ccb627552b138dc872bb0a129"}
Oct 30 21:52:36.791717 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.791664    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podStartSLOduration=1.791650079 podStartE2EDuration="1.791650079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:36.788635154 +0000 UTC m=+2292.076357702" watchObservedRunningTime="2026-10-30 21:52:36.791650079 +0000 UTC m=+2292.079372684"
Oct 30 21:52:36.872872 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:36.872841    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:37.875124 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:37.875092    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 21:52:37.875578 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:37.875155    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 21:52:38.873081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:38.873050    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 21:52:38.873259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:38.873103    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 21:52:39.873930 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:39.873850    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 21:52:39.873930 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:39.873904    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 21:52:40.873054 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:40.873024    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 21:52:40.873227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:40.873072    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 21:52:41.873154 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:41.873117    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 21:52:41.873568 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:41.873167    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 21:52:42.873608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:42.873571    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 21:52:42.874003 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:42.873623    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 21:52:43.873512 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:43.873473    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 21:52:43.873661 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:43.873559    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 21:52:45.950371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:45.949445    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:52:45.950371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:45.949528    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:46.307424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:46.303680    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:46.307424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:46.305291    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:46.345720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:46.345691    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw"
Oct 30 21:52:46.486367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:46.485434    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:46.486367 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:46.485811    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:52:46.610130 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:52:46.610007    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:52:46.722384 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:46.721812    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:46.891431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:46.890671    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:47.290708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.290654    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:47.294739 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.294704    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:47.492371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.487622    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:47.530489 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.530445    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:47.532529 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.532141    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:52:47.539520 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.539494    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:52:47.907371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.891621    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:47.985826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.903885    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/health\": context deadline exceeded"
Oct 30 21:52:47.985826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.915417    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:47.985826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:47.981679    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:52:48.332494 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.332447    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:48.391313 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.389470    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": context deadline exceeded"
Oct 30 21:52:48.427451 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.427393    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:52:48.489901 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.489864    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:48.578724 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.568871    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:52:48.587612 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.587517    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:52:48.610371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.596522    5817 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:48.704816 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.703806    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:48.819429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.819382    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" probeResult="failure" output="Get \"https://10.134.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:48.946405 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.946309    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:52:48.949166 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.949130    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 21:52:48.960596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.960573    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 21:52:48.962703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:48.962685    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 21:52:58.704492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:58.704459    5817 generic.go:356] "Generic (PLEG): container finished" podID="9faf45e5-f84c-483c-8540-01dc105b932b" containerID="206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74" exitCode=0
Oct 30 21:52:58.705142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:58.704520    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" event={"ID":"9faf45e5-f84c-483c-8540-01dc105b932b","Type":"ContainerDied","Data":"206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74"}
Oct 30 21:52:58.705142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:58.704876    5817 scope.go:122] "RemoveContainer" containerID="206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74"
Oct 30 21:52:58.706555 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:58.706518    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:58.706675 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:58.706596    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:58.706675 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:58.706632    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:52:59.730829 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:59.730789    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" event={"ID":"9faf45e5-f84c-483c-8540-01dc105b932b","Type":"ContainerStarted","Data":"b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77"}
Oct 30 21:52:59.765654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:59.765627    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:52:59.766479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:59.766455    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 21:52:59.766616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:52:59.766507    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 21:53:00.766329 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:00.766294    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 21:53:00.766776 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:00.766368    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 21:53:01.766167 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:01.766133    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 21:53:01.766338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:01.766193    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 21:53:02.765976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:02.765942    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 21:53:02.766150 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:02.765993    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 21:53:03.342218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:03.342188    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:53:03.766118 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:03.766085    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 21:53:03.766295 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:03.766137    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 21:53:04.766466 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:04.766429    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 21:53:04.766878 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:04.766492    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 21:53:05.773192 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:05.773153    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:53:05.775182 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:05.775156    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 21:53:16.338813 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:53:16.338771    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:53:18.072067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.070611    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"]
Oct 30 21:53:18.091876 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.091838    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.177692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.177656    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w5-ca0a55-openshift-default-dockercfg-2p2nf\""
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.196108    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"]
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.196394    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-envoy\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.196569    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-socket\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.234458    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-token\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.234583    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-credential-socket\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.234631    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-data\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.234659    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istiod-ca-cert\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.234698    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx5nj\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-kube-api-access-hx5nj\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.234744    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-certs\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.295370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.234807    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-podinfo\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.342990    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-socket\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343082    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-token\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343147    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-credential-socket\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343192    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-data\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343219    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istiod-ca-cert\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343258    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx5nj\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-kube-api-access-hx5nj\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343304    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-certs\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343384    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-podinfo\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.343546    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-envoy\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.353034    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-envoy\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.353315    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-socket\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.354014    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-credential-socket\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.354260    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-data\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.355103    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istiod-ca-cert\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.362418 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.355648    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-certs\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.367950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.367923    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-podinfo\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.419442 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.419409    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx5nj\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-kube-api-access-hx5nj\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.519373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.474201    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-token\") pod \"e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") " pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:18.732937 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:18.732633    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:19.350218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:19.350183    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"]
Oct 30 21:53:19.352667 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:53:19.352628    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8839ba_a89c_43b8_a9eb_c4c96b232b2d.slice/crio-975bb501e41c14144363ec5b11a1c9134234dc32b306623b97a60821f84c1f5b WatchSource:0}: Error finding container 975bb501e41c14144363ec5b11a1c9134234dc32b306623b97a60821f84c1f5b: Status 404 returned error can't find the container with id 975bb501e41c14144363ec5b11a1c9134234dc32b306623b97a60821f84c1f5b
Oct 30 21:53:19.355759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:19.355714    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:19.355833 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:19.355813    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:19.355870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:19.355857    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:20.172700 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:20.171340    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" event={"ID":"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d","Type":"ContainerStarted","Data":"86a49f91ea060a774a03b9d6257eb1b9f13aec532207af6ecb13acdcc7bdeabd"}
Oct 30 21:53:20.172700 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:20.171474    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" event={"ID":"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d","Type":"ContainerStarted","Data":"975bb501e41c14144363ec5b11a1c9134234dc32b306623b97a60821f84c1f5b"}
Oct 30 21:53:20.739755 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:20.739718    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:20.747628 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:20.747598    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:20.869566 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:20.869505    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podStartSLOduration=2.869487365 podStartE2EDuration="2.869487365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:20.400152094 +0000 UTC m=+2335.687874641" watchObservedRunningTime="2026-10-30 21:53:20.869487365 +0000 UTC m=+2336.157209912"
Oct 30 21:53:21.179710 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:21.179624    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:21.180654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:21.180625    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:53:26.964279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:26.964235    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:53:26.964798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:26.964671    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:53:27.024546 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:27.024511    5817 scope.go:122] "RemoveContainer" containerID="c1a96c20f88345383bb0376a21df5aeb56705beb7c018d338c646702490327d1"
Oct 30 21:53:27.288556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:27.288470    5817 patch_prober.go:29] interesting pod/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:53:27.288708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:27.288546    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:53:35.341199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.341165    5817 generic.go:356] "Generic (PLEG): container finished" podID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerID="183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5" exitCode=0
Oct 30 21:53:35.341668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.341238    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" event={"ID":"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc","Type":"ContainerDied","Data":"183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5"}
Oct 30 21:53:35.341722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.341698    5817 scope.go:122] "RemoveContainer" containerID="183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5"
Oct 30 21:53:35.343474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.343441    5817 generic.go:356] "Generic (PLEG): container finished" podID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerID="3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae" exitCode=0
Oct 30 21:53:35.343581 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.343492    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" event={"ID":"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d","Type":"ContainerDied","Data":"3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae"}
Oct 30 21:53:35.343581 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.343504    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:35.343652 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.343589    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:35.343652 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.343635    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:35.343882 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.343863    5817 scope.go:122] "RemoveContainer" containerID="3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae"
Oct 30 21:53:35.345480 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.345455    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:35.345544 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.345528    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:35.345591 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:35.345577    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:36.353396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.353321    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" event={"ID":"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc","Type":"ContainerStarted","Data":"294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e"}
Oct 30 21:53:36.355708 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.355680    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" event={"ID":"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d","Type":"ContainerStarted","Data":"da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633"}
Oct 30 21:53:36.451691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.451651    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:53:36.452087 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.452059    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 21:53:36.452197 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.452124    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 21:53:36.465115 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.465097    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:53:36.465200 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.465181    5817 patch_prober.go:29] interesting pod/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 21:53:36.465255 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:36.465212    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 21:53:37.451766 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:37.451720    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 21:53:37.452275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:37.451791    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 21:53:37.465535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:37.465507    5817 patch_prober.go:29] interesting pod/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 21:53:37.465651 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:37.465557    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 21:53:38.451441 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:38.451398    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 21:53:38.451654 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:38.451503    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 21:53:38.465633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:38.465601    5817 patch_prober.go:29] interesting pod/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 21:53:38.465994 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:38.465647    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 21:53:39.451944 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:39.451914    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 21:53:39.452182 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:39.452153    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 21:53:39.465769 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:39.465739    5817 patch_prober.go:29] interesting pod/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 21:53:39.466206 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:39.466092    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 21:53:40.451847 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:40.451816    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 21:53:40.452178 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:40.452154    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 21:53:40.465122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:40.465100    5817 patch_prober.go:29] interesting pod/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 21:53:40.465290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:40.465271    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 21:53:40.925426 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:40.924441    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:53:41.247275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.247239    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:53:41.451906 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.451878    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 21:53:41.452155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.452119    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 21:53:41.465245 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.465221    5817 patch_prober.go:29] interesting pod/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused" start-of-body=
Oct 30 21:53:41.465374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.465264    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.110:15021/healthz/ready\": dial tcp 10.134.0.110:15021: connect: connection refused"
Oct 30 21:53:41.947279 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.947235    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"]
Oct 30 21:53:41.953154 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.953120    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.978198 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.978153    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-envoy\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.978539 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.978510    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-token\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.978757 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.978735    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-data\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.978963 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.978933    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-certs\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.979174 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.979148    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-credential-socket\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.979396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.979373    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npwr4\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-kube-api-access-npwr4\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.979696 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.979675    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-socket\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.979877 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.979856    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0ddf30-bcf7-4449-8871-9245159c0b09-istiod-ca-cert\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:41.980067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:41.980047    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-podinfo\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.035701 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.035669    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-ed6135-openshift-default-dockercfg-l6xcn\""
Oct 30 21:53:42.066091 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.066051    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"]
Oct 30 21:53:42.174966 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.174918    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-socket\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.177571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.177531    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0ddf30-bcf7-4449-8871-9245159c0b09-istiod-ca-cert\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.177898 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.177867    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-podinfo\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.178079 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.178059    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-envoy\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.178253 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.178234    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-token\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.178452 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.178432    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-data\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.185483 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.178720    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-certs\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.185711 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.185686    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-credential-socket\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.185924 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.185906    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npwr4\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-kube-api-access-npwr4\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.187147 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.187121    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-data\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.194992 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.187679    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-socket\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.195205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.188370    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0ddf30-bcf7-4449-8871-9245159c0b09-istiod-ca-cert\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.195316 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.188631    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-certs\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.195449 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.188879    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-credential-socket\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.212608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.211126    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-podinfo\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.212789 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.212769    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-envoy\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.254371 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.248986    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-token\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.280171 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.278935    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npwr4\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-kube-api-access-npwr4\") pod \"e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") " pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.290649 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.281120    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:42.470175 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.470091    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:53:42.471362 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.471330    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:53:42.538722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.538693    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:53:42.540105 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:42.540078    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:53:43.048586 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:43.048556    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"]
Oct 30 21:53:43.050207 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:53:43.050181    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0ddf30_bcf7_4449_8871_9245159c0b09.slice/crio-c18aa060d0a0907e86a02fdc77736b0fe13bc819ea22912072e7117544c1ddda WatchSource:0}: Error finding container c18aa060d0a0907e86a02fdc77736b0fe13bc819ea22912072e7117544c1ddda: Status 404 returned error can't find the container with id c18aa060d0a0907e86a02fdc77736b0fe13bc819ea22912072e7117544c1ddda
Oct 30 21:53:43.052872 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:43.052838    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:43.052944 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:43.052922    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:43.052981 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:43.052952    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:43.470302 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:43.470264    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" event={"ID":"0a0ddf30-bcf7-4449-8871-9245159c0b09","Type":"ContainerStarted","Data":"429d30769691934013a89accc134839e359b067a75bd1d05be429df09aa78cc5"}
Oct 30 21:53:43.470508 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:43.470319    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" event={"ID":"0a0ddf30-bcf7-4449-8871-9245159c0b09","Type":"ContainerStarted","Data":"c18aa060d0a0907e86a02fdc77736b0fe13bc819ea22912072e7117544c1ddda"}
Oct 30 21:53:43.599266 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:43.599213    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podStartSLOduration=2.599199895 podStartE2EDuration="2.599199895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:43.596215521 +0000 UTC m=+2358.883938067" watchObservedRunningTime="2026-10-30 21:53:43.599199895 +0000 UTC m=+2358.886922444"
Oct 30 21:53:44.291240 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:44.291207    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:44.296609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:44.296583    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:44.477311 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:44.477280    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:44.478320 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:44.478297    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:53:46.339316 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:53:46.339272    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:53:53.537759 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.537725    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"]
Oct 30 21:53:53.542687 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.542663    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.602434 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.602401    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"]
Oct 30 21:53:53.642912 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.642881    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-7d7f57-openshift-default-dockercfg-gd4bf\""
Oct 30 21:53:53.665155 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.665125    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jss99\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-kube-api-access-jss99\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.687440 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.665172    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-credential-socket\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.691372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.687822    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istiod-ca-cert\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.698665 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.698611    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-envoy\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.703779 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.702396    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-data\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.704190 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.704154    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-token\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.704609 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.704572    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-socket\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.711100 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.711071    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-certs\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.711422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.711392    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-podinfo\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813159 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813073    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-socket\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813159 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813131    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-certs\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813227    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-podinfo\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813288    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jss99\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-kube-api-access-jss99\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813327    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-credential-socket\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813420 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813396    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istiod-ca-cert\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813430    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-envoy\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813463    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-socket\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813497    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-data\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.813645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813509    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-certs\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.814157 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813576    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-token\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.814274 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.814195    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istiod-ca-cert\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.814274 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.813705    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-credential-socket\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.814274 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.814068    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-data\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.816150 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.816121    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-podinfo\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.816807 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.816782    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-envoy\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.873606 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.873527    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jss99\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-kube-api-access-jss99\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:53.875490 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:53.875441    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-token\") pod \"e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") " pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:54.160123 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:54.160036    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:54.590541 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:53:54.590502    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dcbc03b_703e_4e1c_a76c_339a0bc312a3.slice/crio-78468b142c2dd0f2ddb1f016b2e1dcfec5c2656b2d458706a41b71a2839e595f WatchSource:0}: Error finding container 78468b142c2dd0f2ddb1f016b2e1dcfec5c2656b2d458706a41b71a2839e595f: Status 404 returned error can't find the container with id 78468b142c2dd0f2ddb1f016b2e1dcfec5c2656b2d458706a41b71a2839e595f
Oct 30 21:53:54.590541 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:54.590530    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"]
Oct 30 21:53:54.593669 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:54.593632    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:54.593805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:54.593713    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:54.593805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:54.593761    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:53:54.665307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:54.665268    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" event={"ID":"4dcbc03b-703e-4e1c-a76c-339a0bc312a3","Type":"ContainerStarted","Data":"78468b142c2dd0f2ddb1f016b2e1dcfec5c2656b2d458706a41b71a2839e595f"}
Oct 30 21:53:55.673315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:55.673276    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" event={"ID":"4dcbc03b-703e-4e1c-a76c-339a0bc312a3","Type":"ContainerStarted","Data":"8194d05fe70d6f5fd20e59c4858c739dc51af51b3b940f7ba613268e5f505e4f"}
Oct 30 21:53:55.838585 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:55.838529    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podStartSLOduration=2.838511901 podStartE2EDuration="2.838511901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:55.835892913 +0000 UTC m=+2371.123615460" watchObservedRunningTime="2026-10-30 21:53:55.838511901 +0000 UTC m=+2371.126234448"
Oct 30 21:53:56.160254 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:56.160211    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:56.166553 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:56.166529    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:56.681273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:56.681245    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:53:56.682252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:53:56.682235    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:54:16.338480 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:16.338439    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:54:17.855438 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.855402    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"]
Oct 30 21:54:17.869473 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.869445    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.922020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.921936    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"]
Oct 30 21:54:17.952241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952206    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-token\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.952447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952364    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istiod-ca-cert\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.952447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952427    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-credential-socket\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.952572 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952533    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-envoy\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.952743 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952707    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-certs\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.952798 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952757    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24sp8\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-kube-api-access-24sp8\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.952864 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952849    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-podinfo\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.952922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.952884    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-socket\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.953081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.953061    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-data\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:17.960412 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:17.960389    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-167561-openshift-default-dockercfg-hdjzl\""
Oct 30 21:54:18.054849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.054814    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-data\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.054888    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-token\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.054934    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istiod-ca-cert\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055026 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.054978    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-credential-socket\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055029    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-envoy\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055151    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-certs\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055205 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055188    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24sp8\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-kube-api-access-24sp8\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055390 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055296    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-podinfo\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055390 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055329    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-socket\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055390 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055340    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-credential-socket\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055575 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055292    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-data\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055572    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-certs\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055696 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055670    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-socket\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.055808 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.055789    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istiod-ca-cert\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.057388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.057324    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-envoy\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.057693 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.057675    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-podinfo\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.115674 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.115600    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-token\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.115875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.115854    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24sp8\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-kube-api-access-24sp8\") pod \"e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") " pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.184272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.184243    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:18.871713 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.871684    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"]
Oct 30 21:54:18.872764 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:54:18.872739    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f91b4f_7f3f_492a_aaea_d71076cde8c5.slice/crio-e2bfde75ee5c13b7917cf6b75bb3aecd18cd920ecc0a86f8262fe37f2984de1c WatchSource:0}: Error finding container e2bfde75ee5c13b7917cf6b75bb3aecd18cd920ecc0a86f8262fe37f2984de1c: Status 404 returned error can't find the container with id e2bfde75ee5c13b7917cf6b75bb3aecd18cd920ecc0a86f8262fe37f2984de1c
Oct 30 21:54:18.875739 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.875703    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:18.875846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.875769    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:18.875846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.875815    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:54:18.992544 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.992509    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" event={"ID":"d4f91b4f-7f3f-492a-aaea-d71076cde8c5","Type":"ContainerStarted","Data":"fa612a3be8b390420799ca6d42c486dd996d1fe882ccfb3b7eaf73ffb7f6c1c7"}
Oct 30 21:54:18.992744 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:18.992560    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" event={"ID":"d4f91b4f-7f3f-492a-aaea-d71076cde8c5","Type":"ContainerStarted","Data":"e2bfde75ee5c13b7917cf6b75bb3aecd18cd920ecc0a86f8262fe37f2984de1c"}
Oct 30 21:54:19.116066 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:19.116012    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podStartSLOduration=2.115993651 podStartE2EDuration="2.115993651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:19.112988143 +0000 UTC m=+2394.400710689" watchObservedRunningTime="2026-10-30 21:54:19.115993651 +0000 UTC m=+2394.403716200"
Oct 30 21:54:19.184903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:19.184830    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:19.186387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:19.186321    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:54:19.186521 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:19.186389    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 21:54:20.190631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:20.190597    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:21.011299 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:21.011266    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:21.012118 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:21.012091    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:54:36.138731 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:36.138696    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"]
Oct 30 21:54:36.184749 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:36.184723    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"]
Oct 30 21:54:36.184966 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:36.184928    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" containerID="cri-o://973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b" gracePeriod=30
Oct 30 21:54:36.208427 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:36.208394    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" containerID="cri-o://e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9" gracePeriod=30
Oct 30 21:54:36.748848 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:36.748803    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"]
Oct 30 21:54:36.749079 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:36.749056    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy" containerID="cri-o://fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535" gracePeriod=30
Oct 30 21:54:37.100509 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:37.100417    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"]
Oct 30 21:54:37.100731 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:37.100688    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy" containerID="cri-o://d0279b942fb7e050d13960e3567c5754cc9e6e835c5c38199062145b017b4fc3" gracePeriod=30
Oct 30 21:54:37.924156 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:37.924081    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"]
Oct 30 21:54:37.924787 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:37.924366    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy" containerID="cri-o://63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01" gracePeriod=30
Oct 30 21:54:38.281441 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:38.281406    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"]
Oct 30 21:54:38.281702 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:38.281661    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" containerID="cri-o://6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4" gracePeriod=30
Oct 30 21:54:38.784076 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:38.784034    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:54:38.784259 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:38.784095    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:54:40.077044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:40.077007    5817 patch_prober.go:29] interesting pod/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:54:40.077441 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:40.077089    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:54:41.525852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.525823    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:54:41.572171 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572144    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-kube-api-access-n9kk2\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-kube-api-access-n9kk2\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572245    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-token\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-token\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572284    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-data\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572377    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-certs\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572439    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-socket\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572569 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572486    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-envoy\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572569 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572520    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-podinfo\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572666    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istiod-ca-cert\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572745    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-certs" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.572950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572781    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-credential-socket\") pod \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\" (UID: \"7db80d85-129d-4d02-b23b-1be8eaa5d1fe\") "
Oct 30 21:54:41.572950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.572831    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-socket" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.573253 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573120    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istiod-ca-cert" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:41.573358 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573325    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-credential-socket" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.574562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573634    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-data" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.574562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573711    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.574562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573732    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.574562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573746    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.574562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573761    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.574562 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.573865    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:54:41.575797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.575774    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-kube-api-access-n9kk2" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "kube-api-access-n9kk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:41.575897 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.575849    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-token" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:41.576741 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.576701    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-envoy" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.577671 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.577651    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-podinfo" pod "7db80d85-129d-4d02-b23b-1be8eaa5d1fe" (UID: "7db80d85-129d-4d02-b23b-1be8eaa5d1fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:41.674590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674556    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-credential-socket\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.674784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674612    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-kube-api-access-7czmq\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-kube-api-access-7czmq\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.674784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674676    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-socket\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.674784 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674704    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-podinfo\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.674949 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674789    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/adfa29b9-06f0-4a23-b49e-154decfb13db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adfa29b9-06f0-4a23-b49e-154decfb13db-istiod-ca-cert\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.674949 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674823    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-token\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-token\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.674949 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674884    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-credential-socket" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.674949 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674905    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-data\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.675168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.674975    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-envoy\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.675168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675015    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-certs\") pod \"adfa29b9-06f0-4a23-b49e-154decfb13db\" (UID: \"adfa29b9-06f0-4a23-b49e-154decfb13db\") "
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675012    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-socket" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675242    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adfa29b9-06f0-4a23-b49e-154decfb13db-istiod-ca-cert" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675265    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-data" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675936    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675958    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675975    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.675989    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.676001    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.676015    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.676700    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adfa29b9-06f0-4a23-b49e-154decfb13db-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.676735    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9kk2\" (UniqueName: \"kubernetes.io/projected/7db80d85-129d-4d02-b23b-1be8eaa5d1fe-kube-api-access-n9kk2\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.676771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.676752    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.677231 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.677208    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-certs" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.678035 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.677976    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-envoy" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:41.678135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.678096    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-podinfo" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:41.678135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.678113    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-token" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:41.678257 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.678138    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-kube-api-access-7czmq" pod "adfa29b9-06f0-4a23-b49e-154decfb13db" (UID: "adfa29b9-06f0-4a23-b49e-154decfb13db"). InnerVolumeSpecName "kube-api-access-7czmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:41.779102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.778734    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.779102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.778772    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adfa29b9-06f0-4a23-b49e-154decfb13db-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.779102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.778785    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7czmq\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-kube-api-access-7czmq\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.779102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.778800    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:41.779102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:41.778814    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adfa29b9-06f0-4a23-b49e-154decfb13db-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.026039 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.026015    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:54:42.086597 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086507    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-kube-api-access-xmkpb\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-kube-api-access-xmkpb\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.086770 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086599    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-data\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.086770 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086653    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istiod-ca-cert\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.086770 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086742    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-envoy\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.086933 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086800    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-token\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-token\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.086933 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086864    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-credential-socket\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.086933 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086914    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-data" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.087090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.086976    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-socket\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.087090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087021    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-certs\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.087090 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087080    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-podinfo\") pod \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\" (UID: \"10fcc6c1-f818-4dd2-a7e2-b57296a962eb\") "
Oct 30 21:54:42.087245 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087126    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istiod-ca-cert" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:42.087333 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087257    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-credential-socket" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.087417 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087330    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-certs" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.087530 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087508    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-socket" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.087858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087830    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.087858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087859    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.088053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087874    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.088053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087889    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.088053 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.087903    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.089293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.089270    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-kube-api-access-xmkpb" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "kube-api-access-xmkpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:42.089695 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.089673    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-token" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:42.089822 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.089803    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-envoy" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.089865 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.089847    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-podinfo" pod "10fcc6c1-f818-4dd2-a7e2-b57296a962eb" (UID: "10fcc6c1-f818-4dd2-a7e2-b57296a962eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:42.188955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.188917    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.188955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.188952    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmkpb\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-kube-api-access-xmkpb\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.188955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.188962    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.189186 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.188970    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10fcc6c1-f818-4dd2-a7e2-b57296a962eb-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.189777 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:42.189749    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d09d74d_9e52_4e28_a306_f429b2bb0a65.slice/crio-d0279b942fb7e050d13960e3567c5754cc9e6e835c5c38199062145b017b4fc3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d09d74d_9e52_4e28_a306_f429b2bb0a65.slice/crio-conmon-d0279b942fb7e050d13960e3567c5754cc9e6e835c5c38199062145b017b4fc3.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:54:42.275651 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.275614    5817 generic.go:356] "Generic (PLEG): container finished" podID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerID="973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b" exitCode=0
Oct 30 21:54:42.275850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.275690    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" event={"ID":"7db80d85-129d-4d02-b23b-1be8eaa5d1fe","Type":"ContainerDied","Data":"973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b"}
Oct 30 21:54:42.275850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.275754    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9" event={"ID":"7db80d85-129d-4d02-b23b-1be8eaa5d1fe","Type":"ContainerDied","Data":"1a3650557de808cab73de199c43be1ed38182b6138084b0a4603c13cb8083443"}
Oct 30 21:54:42.275850 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.275783    5817 scope.go:122] "RemoveContainer" containerID="973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b"
Oct 30 21:54:42.276294 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.276273    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"
Oct 30 21:54:42.278455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.277923    5817 generic.go:356] "Generic (PLEG): container finished" podID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerID="e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9" exitCode=0
Oct 30 21:54:42.278455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.278032    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" event={"ID":"adfa29b9-06f0-4a23-b49e-154decfb13db","Type":"ContainerDied","Data":"e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9"}
Oct 30 21:54:42.278455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.278078    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv" event={"ID":"adfa29b9-06f0-4a23-b49e-154decfb13db","Type":"ContainerDied","Data":"b6ea99788eefa0050e7a7ade62387053da12f3b8be5b72697309b407a6589ecf"}
Oct 30 21:54:42.278455 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.278080    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"
Oct 30 21:54:42.279651 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.279623    5817 generic.go:356] "Generic (PLEG): container finished" podID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerID="fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535" exitCode=0
Oct 30 21:54:42.279765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.279650    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" event={"ID":"10fcc6c1-f818-4dd2-a7e2-b57296a962eb","Type":"ContainerDied","Data":"fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535"}
Oct 30 21:54:42.279765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.279700    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6" event={"ID":"10fcc6c1-f818-4dd2-a7e2-b57296a962eb","Type":"ContainerDied","Data":"765f5dd6b061e63e18c36177133864be240cc26e6c76ca4e1eb5004c23d16b9f"}
Oct 30 21:54:42.279765 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.279661    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"
Oct 30 21:54:42.281667 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.281649    5817 generic.go:356] "Generic (PLEG): container finished" podID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerID="d0279b942fb7e050d13960e3567c5754cc9e6e835c5c38199062145b017b4fc3" exitCode=0
Oct 30 21:54:42.281764 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.281721    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" event={"ID":"0d09d74d-9e52-4e28-a306-f429b2bb0a65","Type":"ContainerDied","Data":"d0279b942fb7e050d13960e3567c5754cc9e6e835c5c38199062145b017b4fc3"}
Oct 30 21:54:42.291595 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.289970    5817 scope.go:122] "RemoveContainer" containerID="973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b"
Oct 30 21:54:42.291595 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:42.290439    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b\": container with ID starting with 973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b not found: ID does not exist" containerID="973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b"
Oct 30 21:54:42.291595 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.290465    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b"} err="failed to get container status \"973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b\": rpc error: code = NotFound desc = could not find container \"973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b\": container with ID starting with 973e40f8fe8a513ae2fa279de423bcd2d3cb655313adf4c6f8d4372ef813a57b not found: ID does not exist"
Oct 30 21:54:42.291595 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.290488    5817 scope.go:122] "RemoveContainer" containerID="e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9"
Oct 30 21:54:42.318048 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.317796    5817 scope.go:122] "RemoveContainer" containerID="e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9"
Oct 30 21:54:42.318181 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:42.318157    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9\": container with ID starting with e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9 not found: ID does not exist" containerID="e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9"
Oct 30 21:54:42.318241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.318195    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9"} err="failed to get container status \"e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9\": rpc error: code = NotFound desc = could not find container \"e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9\": container with ID starting with e2513513791c6d1ebc9f5db0526a34af6cbee0e309de383902cd202fb98bc5d9 not found: ID does not exist"
Oct 30 21:54:42.318241 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.318222    5817 scope.go:122] "RemoveContainer" containerID="fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535"
Oct 30 21:54:42.332303 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.332275    5817 scope.go:122] "RemoveContainer" containerID="fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535"
Oct 30 21:54:42.332687 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:42.332658    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535\": container with ID starting with fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535 not found: ID does not exist" containerID="fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535"
Oct 30 21:54:42.332779 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.332697    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535"} err="failed to get container status \"fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535\": rpc error: code = NotFound desc = could not find container \"fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535\": container with ID starting with fba4adab009c4b7340128708948af1668042a2d7b0f725bfd67b41d67297e535 not found: ID does not exist"
Oct 30 21:54:42.397738 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.397707    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:54:42.436903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.436851    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"]
Oct 30 21:54:42.449391 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.449334    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wvmr9"]
Oct 30 21:54:42.492639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.492604    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-socket\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.492805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.492707    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-token\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-token\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.492805 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.492748    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-data\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.492887 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.492847    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-credential-socket\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.492920 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.492884    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-socket" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.492954 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.492924    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-podinfo\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.492985 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.492966    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-kube-api-access-7qvqb\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-kube-api-access-7qvqb\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.493076 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.493056    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-credential-socket" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.493140 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.493107    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-data" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.493207 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.493182    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-certs\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.493265 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.493224    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istiod-ca-cert\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.493322 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.493272    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-envoy\") pod \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\" (UID: \"0d09d74d-9e52-4e28-a306-f429b2bb0a65\") "
Oct 30 21:54:42.493499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.493477    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-certs" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.493672 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.493648    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istiod-ca-cert" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:42.494059 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.494038    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.494059 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.494058    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.494227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.494068    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.494227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.494076    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.494227 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.494086    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.495499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.495469    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-kube-api-access-7qvqb" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "kube-api-access-7qvqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:42.495499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.495476    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-envoy" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:42.495616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.495558    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-podinfo" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:42.495668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.495631    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-token" pod "0d09d74d-9e52-4e28-a306-f429b2bb0a65" (UID: "0d09d74d-9e52-4e28-a306-f429b2bb0a65"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:42.555307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.555265    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"]
Oct 30 21:54:42.571044 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.571002    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-wfjsv"]
Oct 30 21:54:42.596070 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.596032    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.596070 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.596068    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qvqb\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-kube-api-access-7qvqb\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.596327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.596086    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.596327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.596101    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d09d74d-9e52-4e28-a306-f429b2bb0a65-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.679377 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.679252    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"]
Oct 30 21:54:42.688553 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.688512    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-cgxs6"]
Oct 30 21:54:42.849036 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.848999    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" path="/var/lib/kubelet/pods/10fcc6c1-f818-4dd2-a7e2-b57296a962eb/volumes"
Oct 30 21:54:42.849826 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.849800    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" path="/var/lib/kubelet/pods/7db80d85-129d-4d02-b23b-1be8eaa5d1fe/volumes"
Oct 30 21:54:42.850496 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:42.850474    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" path="/var/lib/kubelet/pods/adfa29b9-06f0-4a23-b49e-154decfb13db/volumes"
Oct 30 21:54:43.224767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.224742    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:54:43.292061 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.292028    5817 generic.go:356] "Generic (PLEG): container finished" podID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerID="63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01" exitCode=0
Oct 30 21:54:43.292246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.292081    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"
Oct 30 21:54:43.292246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.292110    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" event={"ID":"94a1e08c-32a1-420c-8187-1f631b3f7077","Type":"ContainerDied","Data":"63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01"}
Oct 30 21:54:43.292246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.292163    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p" event={"ID":"94a1e08c-32a1-420c-8187-1f631b3f7077","Type":"ContainerDied","Data":"611fd032c4df611514a0f8293d18d5b02e92aa6bf367f7fdd84417204288a7ef"}
Oct 30 21:54:43.292246 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.292190    5817 scope.go:122] "RemoveContainer" containerID="63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01"
Oct 30 21:54:43.296046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.296020    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z" event={"ID":"0d09d74d-9e52-4e28-a306-f429b2bb0a65","Type":"ContainerDied","Data":"64acfe0b9911ac4990483eab8250bd7e85d89781139aa437bae167af8ee1c282"}
Oct 30 21:54:43.296153 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.296057    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"
Oct 30 21:54:43.308737 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.308715    5817 scope.go:122] "RemoveContainer" containerID="63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01"
Oct 30 21:54:43.309055 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:43.309032    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01\": container with ID starting with 63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01 not found: ID does not exist" containerID="63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01"
Oct 30 21:54:43.309120 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309064    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01"} err="failed to get container status \"63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01\": rpc error: code = NotFound desc = could not find container \"63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01\": container with ID starting with 63a073ef1438e7c4bc63a64f207ae62c93913f6069008df8d7e46081ff8aad01 not found: ID does not exist"
Oct 30 21:54:43.309120 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309082    5817 scope.go:122] "RemoveContainer" containerID="d0279b942fb7e050d13960e3567c5754cc9e6e835c5c38199062145b017b4fc3"
Oct 30 21:54:43.309216 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309134    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-socket\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.309216 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309199    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-data\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.309395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309372    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-socket" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.309486 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309463    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-data" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.309639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309621    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94a1e08c-32a1-420c-8187-1f631b3f7077-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a1e08c-32a1-420c-8187-1f631b3f7077-istiod-ca-cert\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.309721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309652    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-certs\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.309721 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309704    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-podinfo\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.309820 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309746    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-token\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-token\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.309820 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309784    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-kube-api-access-vfg6t\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-kube-api-access-vfg6t\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.310016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309857    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-credential-socket\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.310016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309885    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-envoy\") pod \"94a1e08c-32a1-420c-8187-1f631b3f7077\" (UID: \"94a1e08c-32a1-420c-8187-1f631b3f7077\") "
Oct 30 21:54:43.310016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.309992    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-certs" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.310177 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.310011    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94a1e08c-32a1-420c-8187-1f631b3f7077-istiod-ca-cert" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:43.310229 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.310204    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-credential-socket" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.310710 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.310685    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94a1e08c-32a1-420c-8187-1f631b3f7077-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.310710 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.310712    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.310879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.310724    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.310879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.310736    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.310879 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.310747    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.312038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.312010    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-kube-api-access-vfg6t" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "kube-api-access-vfg6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:43.312131 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.312106    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-podinfo" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:43.312644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.312627    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-token" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:43.312830 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.312796    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-envoy" pod "94a1e08c-32a1-420c-8187-1f631b3f7077" (UID: "94a1e08c-32a1-420c-8187-1f631b3f7077"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.386250 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.386216    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 21:54:43.386493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.386290    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 21:54:43.412296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.412273    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.412296 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.412296    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.412485 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.412306    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfg6t\" (UniqueName: \"kubernetes.io/projected/94a1e08c-32a1-420c-8187-1f631b3f7077-kube-api-access-vfg6t\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.412485 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.412316    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94a1e08c-32a1-420c-8187-1f631b3f7077-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.501915 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.501878    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"]
Oct 30 21:54:43.512638 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.512602    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-lkh9z"]
Oct 30 21:54:43.571388 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.571361    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:54:43.615064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615037    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-envoy\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615181 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615079    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-certs\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615181 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615148    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-socket\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615254 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615208    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istiod-ca-cert\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615254 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615243    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-podinfo\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615358 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615286    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-credential-socket\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615415 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615390    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-data\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615465 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615449    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-token\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-token\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615675    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-kube-api-access-r48gf\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-kube-api-access-r48gf\") pod \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\" (UID: \"29fba7fc-aa17-4db1-9d8b-3ac5d0084483\") "
Oct 30 21:54:43.615703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615430    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-certs" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.615875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615496    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-socket" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.615875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615636    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-credential-socket" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.615875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615686    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-data" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.615875 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.615729    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istiod-ca-cert" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:43.616544 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.616518    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.616662 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.616547    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.616662 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.616562    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.616662 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.616576    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.616662 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.616589    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.631848 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.631816    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-podinfo" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:43.631957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.631814    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-envoy" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:43.631957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.631850    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-kube-api-access-r48gf" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "kube-api-access-r48gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:43.631957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.631880    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-token" pod "29fba7fc-aa17-4db1-9d8b-3ac5d0084483" (UID: "29fba7fc-aa17-4db1-9d8b-3ac5d0084483"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:43.718293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.718200    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.718293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.718235    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.718293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.718249    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.718293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.718261    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r48gf\" (UniqueName: \"kubernetes.io/projected/29fba7fc-aa17-4db1-9d8b-3ac5d0084483-kube-api-access-r48gf\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:43.800649 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.800611    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"]
Oct 30 21:54:43.812043 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:43.811969    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-sjv9p"]
Oct 30 21:54:44.306465 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.306430    5817 generic.go:356] "Generic (PLEG): container finished" podID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerID="6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4" exitCode=0
Oct 30 21:54:44.306657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.306475    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"
Oct 30 21:54:44.306657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.306512    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" event={"ID":"29fba7fc-aa17-4db1-9d8b-3ac5d0084483","Type":"ContainerDied","Data":"6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4"}
Oct 30 21:54:44.306657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.306570    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw" event={"ID":"29fba7fc-aa17-4db1-9d8b-3ac5d0084483","Type":"ContainerDied","Data":"9f96e4dbf178cbecbba162d92673e7e613ef7271c2367c6b13ef7aefe337db00"}
Oct 30 21:54:44.306657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.306598    5817 scope.go:122] "RemoveContainer" containerID="6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4"
Oct 30 21:54:44.318477 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.318460    5817 scope.go:122] "RemoveContainer" containerID="6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4"
Oct 30 21:54:44.318782 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:44.318758    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4\": container with ID starting with 6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4 not found: ID does not exist" containerID="6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4"
Oct 30 21:54:44.318882 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.318787    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4"} err="failed to get container status \"6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4\": rpc error: code = NotFound desc = could not find container \"6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4\": container with ID starting with 6d909d8c8b4eb75ed9ec80dfa28debf834c023be8994ae9f8acfe8a91daaf6a4 not found: ID does not exist"
Oct 30 21:54:44.469413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.469332    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"]
Oct 30 21:54:44.473777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.473751    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-62gkw"]
Oct 30 21:54:44.846813 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.846779    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" path="/var/lib/kubelet/pods/0d09d74d-9e52-4e28-a306-f429b2bb0a65/volumes"
Oct 30 21:54:44.847474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.847444    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" path="/var/lib/kubelet/pods/29fba7fc-aa17-4db1-9d8b-3ac5d0084483/volumes"
Oct 30 21:54:44.847991 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:44.847969    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" path="/var/lib/kubelet/pods/94a1e08c-32a1-420c-8187-1f631b3f7077/volumes"
Oct 30 21:54:46.340733 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:54:46.340695    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:54:57.159307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:57.159265    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"]
Oct 30 21:54:57.263118 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:54:57.263083    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"]
Oct 30 21:55:01.920431 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.920398    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"]
Oct 30 21:55:01.921081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921066    5817 cpu_manager.go:414] "containerMap: removing container" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy"
Oct 30 21:55:01.921126 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921083    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy"
Oct 30 21:55:01.921126 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921096    5817 cpu_manager.go:414] "containerMap: removing container" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy"
Oct 30 21:55:01.921126 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921101    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy"
Oct 30 21:55:01.921126 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921121    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy"
Oct 30 21:55:01.921126 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921127    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy"
Oct 30 21:55:01.921285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921137    5817 cpu_manager.go:414] "containerMap: removing container" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy"
Oct 30 21:55:01.921285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921141    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy"
Oct 30 21:55:01.921285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921163    5817 cpu_manager.go:414] "containerMap: removing container" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy"
Oct 30 21:55:01.921285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921168    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy"
Oct 30 21:55:01.921285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921180    5817 cpu_manager.go:414] "containerMap: removing container" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy"
Oct 30 21:55:01.921285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921185    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy"
Oct 30 21:55:01.921488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921302    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d09d74d-9e52-4e28-a306-f429b2bb0a65" containerName="istio-proxy"
Oct 30 21:55:01.921488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921313    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="10fcc6c1-f818-4dd2-a7e2-b57296a962eb" containerName="istio-proxy"
Oct 30 21:55:01.921488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921323    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="29fba7fc-aa17-4db1-9d8b-3ac5d0084483" containerName="istio-proxy"
Oct 30 21:55:01.921488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921330    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="94a1e08c-32a1-420c-8187-1f631b3f7077" containerName="istio-proxy"
Oct 30 21:55:01.921488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921340    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="7db80d85-129d-4d02-b23b-1be8eaa5d1fe" containerName="istio-proxy"
Oct 30 21:55:01.921488 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.921361    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="adfa29b9-06f0-4a23-b49e-154decfb13db" containerName="istio-proxy"
Oct 30 21:55:01.924034 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.924017    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:01.934323 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.934296    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv26c\" (UniqueName: \"kubernetes.io/projected/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-kube-api-access-zv26c\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:01.934694 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.934667    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:01.934783 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.934754    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:01.934976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.934959    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:01.978560 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:01.978529    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"]
Oct 30 21:55:02.036338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.036304    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv26c\" (UniqueName: \"kubernetes.io/projected/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-kube-api-access-zv26c\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.036510 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.036421    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.036510 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.036455    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.036587 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.036509    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.037554 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.037523    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.039091 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.039068    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.041105 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.041087    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-ed6135-serving-cert\""
Oct 30 21:55:02.047303 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:02.047016    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-serving-cert: secret "maas-api-e2e-shared-b-w3-ed6135-serving-cert" not found
Oct 30 21:55:02.047303 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:02.047096    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls podName:5c2bb1d9-1a99-4812-9aff-50f0ece9ad09 nodeName:}" failed. No retries permitted until 2026-10-30 21:55:02.547076189 +0000 UTC m=+2437.834798730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls") pod "maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2" (UID: "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09") : secret "maas-api-e2e-shared-b-w3-ed6135-serving-cert" not found
Oct 30 21:55:02.116161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.116125    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv26c\" (UniqueName: \"kubernetes.io/projected/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-kube-api-access-zv26c\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.160159 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.160120    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" podUID="99659e40-3d4e-42ad-83a6-310398106827" containerName="payload-processing" containerID="cri-o://10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556" gracePeriod=25
Oct 30 21:55:02.263445 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.263380    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" podUID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" containerName="payload-pre-processing" containerID="cri-o://aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277" gracePeriod=25
Oct 30 21:55:02.645184 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.645099    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.647630 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.647600    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:02.837280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:02.837240    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:03.243341 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:03.243302    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"]
Oct 30 21:55:03.244734 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:55:03.244708    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2bb1d9_1a99_4812_9aff_50f0ece9ad09.slice/crio-b15aebb7c2aa45045f3ab785cc94dcd23a80255697d18415a7b29eda83654ef6 WatchSource:0}: Error finding container b15aebb7c2aa45045f3ab785cc94dcd23a80255697d18415a7b29eda83654ef6: Status 404 returned error can't find the container with id b15aebb7c2aa45045f3ab785cc94dcd23a80255697d18415a7b29eda83654ef6
Oct 30 21:55:03.461681 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:03.461647    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2" event={"ID":"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09","Type":"ContainerStarted","Data":"b15aebb7c2aa45045f3ab785cc94dcd23a80255697d18415a7b29eda83654ef6"}
Oct 30 21:55:04.470447 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:04.470407    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2" event={"ID":"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09","Type":"ContainerStarted","Data":"dfb610cfc413e6b07bb8e136e6179677702084c781f0d17856f6b055f82ce982"}
Oct 30 21:55:04.470799 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:04.470508    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:04.585852 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:04.585802    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2" podStartSLOduration=2.657562468 podStartE2EDuration="3.585787865s" totalImagesPullingTime="928.225397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:04.585172952 +0000 UTC m=+2439.872895499" watchObservedRunningTime="2026-10-30 21:55:04.585787865 +0000 UTC m=+2439.873510411"
Oct 30 21:55:06.547409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:06.547373    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" podUID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:06.547851 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:06.547410    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" podUID="99659e40-3d4e-42ad-83a6-310398106827" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:07.220494 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:07.220456    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:07.220675 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:07.220529    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:10.483657 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:10.483623    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 21:55:10.520050 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:10.520018    5817 generic.go:356] "Generic (PLEG): container finished" podID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerID="86a49f91ea060a774a03b9d6257eb1b9f13aec532207af6ecb13acdcc7bdeabd" exitCode=0
Oct 30 21:55:10.520217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:10.520052    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" event={"ID":"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d","Type":"ContainerDied","Data":"86a49f91ea060a774a03b9d6257eb1b9f13aec532207af6ecb13acdcc7bdeabd"}
Oct 30 21:55:10.520518 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:10.520499    5817 scope.go:122] "RemoveContainer" containerID="86a49f91ea060a774a03b9d6257eb1b9f13aec532207af6ecb13acdcc7bdeabd"
Oct 30 21:55:10.522077 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:10.522048    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:10.522139 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:10.522122    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:10.522179 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:10.522156    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:11.531235 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:11.531197    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" event={"ID":"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d","Type":"ContainerStarted","Data":"47bdaa4c3b0260e7397cd814c97890c5089b9bdb51ba008584d8545bf5899bd9"}
Oct 30 21:55:11.735168 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:11.735046    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:55:11.735387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:11.735183    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:55:11.735387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:11.735235    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:55:12.734950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:12.734916    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:55:12.735411 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:12.734967    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:55:13.734957 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:13.734920    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:55:13.735378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:13.734997    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:55:14.734841 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:14.734806    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:55:14.735022 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:14.734861    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:55:15.734217 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:15.734178    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:55:15.734424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:15.734234    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:55:16.358627 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:16.358590    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:55:16.546640 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:16.546605    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" podUID="99659e40-3d4e-42ad-83a6-310398106827" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:16.546640 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:16.546635    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" podUID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:55:16.735097 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:16.735057    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused" start-of-body=
Oct 30 21:55:16.735698 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:16.735120    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.112:15021/healthz/ready\": dial tcp 10.134.0.112:15021: connect: connection refused"
Oct 30 21:55:17.738768 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:17.738737    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:55:17.739231 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:17.739210    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:55:17.739860 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:17.739843    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 21:55:20.617111 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.617076    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"]
Oct 30 21:55:20.620535 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.620516    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.669886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.669858    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-7d7f57-serving-cert\""
Oct 30 21:55:20.720989 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.720956    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"]
Oct 30 21:55:20.781072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.781040    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.781304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.781283    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.781407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.781321    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.781517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.781500    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrls5\" (UniqueName: \"kubernetes.io/projected/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-kube-api-access-wrls5\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.883866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.883769    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.884527 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.884409    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.884796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.884761    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.885374 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:20.884981    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-serving-cert: secret "maas-api-e2e-shared-a-w4-7d7f57-serving-cert" not found
Oct 30 21:55:20.885374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.885020    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrls5\" (UniqueName: \"kubernetes.io/projected/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-kube-api-access-wrls5\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.885374 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:20.885058    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls podName:b7cad3a9-6764-4474-9e26-d8e5b9abc0e9 nodeName:}" failed. No retries permitted until 2026-10-30 21:55:21.38503616 +0000 UTC m=+2456.672758709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls") pod "maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25" (UID: "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9") : secret "maas-api-e2e-shared-a-w4-7d7f57-serving-cert" not found
Oct 30 21:55:20.885773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.885749    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.888400 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.888373    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:20.944064 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:20.944029    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrls5\" (UniqueName: \"kubernetes.io/projected/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-kube-api-access-wrls5\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:21.402464 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:21.402427    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:21.404864 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:21.404835    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:21.533732 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:21.533692    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:21.918934 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:21.918901    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"]
Oct 30 21:55:21.920299 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:55:21.920271    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cad3a9_6764_4474_9e26_d8e5b9abc0e9.slice/crio-bfc6248a8e2dfb5fd81bb6e3848bce685feaa6c95999091167528c2ce31d6b46 WatchSource:0}: Error finding container bfc6248a8e2dfb5fd81bb6e3848bce685feaa6c95999091167528c2ce31d6b46: Status 404 returned error can't find the container with id bfc6248a8e2dfb5fd81bb6e3848bce685feaa6c95999091167528c2ce31d6b46
Oct 30 21:55:22.478796 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.478757    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:55:22.533210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.533184    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:55:22.622408 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.622322    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99659e40-3d4e-42ad-83a6-310398106827-kube-api-access-jlr9z\" (UniqueName: \"kubernetes.io/projected/99659e40-3d4e-42ad-83a6-310398106827-kube-api-access-jlr9z\") pod \"99659e40-3d4e-42ad-83a6-310398106827\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") "
Oct 30 21:55:22.622722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.622691    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99659e40-3d4e-42ad-83a6-310398106827-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99659e40-3d4e-42ad-83a6-310398106827-plugins-config-volume\") pod \"99659e40-3d4e-42ad-83a6-310398106827\" (UID: \"99659e40-3d4e-42ad-83a6-310398106827\") "
Oct 30 21:55:22.624055 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.624022    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99659e40-3d4e-42ad-83a6-310398106827-plugins-config-volume" pod "99659e40-3d4e-42ad-83a6-310398106827" (UID: "99659e40-3d4e-42ad-83a6-310398106827"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:22.643002 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.642960    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99659e40-3d4e-42ad-83a6-310398106827-kube-api-access-jlr9z" pod "99659e40-3d4e-42ad-83a6-310398106827" (UID: "99659e40-3d4e-42ad-83a6-310398106827"). InnerVolumeSpecName "kube-api-access-jlr9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:22.643634 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.643588    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25" event={"ID":"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9","Type":"ContainerStarted","Data":"bfc6248a8e2dfb5fd81bb6e3848bce685feaa6c95999091167528c2ce31d6b46"}
Oct 30 21:55:22.645872 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.645846    5817 generic.go:356] "Generic (PLEG): container finished" podID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" containerID="aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277" exitCode=0
Oct 30 21:55:22.646010 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.645957    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"
Oct 30 21:55:22.646086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.646036    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" event={"ID":"14a4a5ad-8af5-4a47-8b77-1ffa94872da0","Type":"ContainerDied","Data":"aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277"}
Oct 30 21:55:22.646086 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.646074    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95" event={"ID":"14a4a5ad-8af5-4a47-8b77-1ffa94872da0","Type":"ContainerDied","Data":"b0969e5b4d0268429031b6eb1cdb2075e39358de5349701c3d44265f0956b925"}
Oct 30 21:55:22.646193 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.646096    5817 scope.go:122] "RemoveContainer" containerID="aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277"
Oct 30 21:55:22.648733 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.648705    5817 generic.go:356] "Generic (PLEG): container finished" podID="99659e40-3d4e-42ad-83a6-310398106827" containerID="10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556" exitCode=0
Oct 30 21:55:22.648998 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.648815    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"
Oct 30 21:55:22.649767 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.648834    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" event={"ID":"99659e40-3d4e-42ad-83a6-310398106827","Type":"ContainerDied","Data":"10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556"}
Oct 30 21:55:22.649869 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.649805    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv" event={"ID":"99659e40-3d4e-42ad-83a6-310398106827","Type":"ContainerDied","Data":"dc284ef2ecc12990194d7018430cfd930abef4b817ccff0365b62c92b19cda43"}
Oct 30 21:55:22.667257 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.667232    5817 scope.go:122] "RemoveContainer" containerID="aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277"
Oct 30 21:55:22.667666 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:22.667637    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277\": container with ID starting with aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277 not found: ID does not exist" containerID="aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277"
Oct 30 21:55:22.667786 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.667674    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277"} err="failed to get container status \"aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277\": rpc error: code = NotFound desc = could not find container \"aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277\": container with ID starting with aa3592591dea7c7d3826bf39b3a48b6bf245e7cc6b9662420cfc7ca095eb7277 not found: ID does not exist"
Oct 30 21:55:22.667786 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.667691    5817 scope.go:122] "RemoveContainer" containerID="10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556"
Oct 30 21:55:22.678956 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.678938    5817 scope.go:122] "RemoveContainer" containerID="10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556"
Oct 30 21:55:22.679265 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:22.679238    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556\": container with ID starting with 10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556 not found: ID does not exist" containerID="10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556"
Oct 30 21:55:22.679310 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.679269    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556"} err="failed to get container status \"10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556\": rpc error: code = NotFound desc = could not find container \"10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556\": container with ID starting with 10b95bec1f8c829d9ab90f17639568ee41974f38c426ef8e899606e434cd7556 not found: ID does not exist"
Oct 30 21:55:22.725478 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.725432    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-plugins-config-volume\") pod \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") "
Oct 30 21:55:22.725840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.725812    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-plugins-config-volume" pod "14a4a5ad-8af5-4a47-8b77-1ffa94872da0" (UID: "14a4a5ad-8af5-4a47-8b77-1ffa94872da0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:22.726135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.726105    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-kube-api-access-xrrjd\" (UniqueName: \"kubernetes.io/projected/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-kube-api-access-xrrjd\") pod \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\" (UID: \"14a4a5ad-8af5-4a47-8b77-1ffa94872da0\") "
Oct 30 21:55:22.726909 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.726882    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlr9z\" (UniqueName: \"kubernetes.io/projected/99659e40-3d4e-42ad-83a6-310398106827-kube-api-access-jlr9z\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:22.727030 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.726916    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99659e40-3d4e-42ad-83a6-310398106827-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:22.727030 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.726931    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:22.728318 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.728292    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-kube-api-access-xrrjd" pod "14a4a5ad-8af5-4a47-8b77-1ffa94872da0" (UID: "14a4a5ad-8af5-4a47-8b77-1ffa94872da0"). InnerVolumeSpecName "kube-api-access-xrrjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:22.815372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.815327    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"]
Oct 30 21:55:22.819283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.819252    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bfdf30b9-66766fc856-9j7zv"]
Oct 30 21:55:22.828944 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.828915    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrrjd\" (UniqueName: \"kubernetes.io/projected/14a4a5ad-8af5-4a47-8b77-1ffa94872da0-kube-api-access-xrrjd\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:22.846142 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:22.846109    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99659e40-3d4e-42ad-83a6-310398106827" path="/var/lib/kubelet/pods/99659e40-3d4e-42ad-83a6-310398106827/volumes"
Oct 30 21:55:23.060410 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:23.060376    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"]
Oct 30 21:55:23.063986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:23.063951    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bfdf30b9-6c99dcd5f5-lwp95"]
Oct 30 21:55:23.659942 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:23.659896    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25" event={"ID":"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9","Type":"ContainerStarted","Data":"390ad2d68a08668a110c297509b62f7c7203f8cd662211f751c935a3230ab7ba"}
Oct 30 21:55:23.660152 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:23.659963    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:23.775976 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:23.775921    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25" podStartSLOduration=2.737524574 podStartE2EDuration="3.775904584s" totalImagesPullingTime="1.03838001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:23.774543551 +0000 UTC m=+2459.062266094" watchObservedRunningTime="2026-10-30 21:55:23.775904584 +0000 UTC m=+2459.063627127"
Oct 30 21:55:24.846072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:24.846041    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" path="/var/lib/kubelet/pods/14a4a5ad-8af5-4a47-8b77-1ffa94872da0/volumes"
Oct 30 21:55:29.674803 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:29.674775    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:55:34.913823 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.913793    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"]
Oct 30 21:55:34.914593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.914564    5817 cpu_manager.go:414] "containerMap: removing container" podUID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" containerName="payload-pre-processing"
Oct 30 21:55:34.914593 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.914594    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" containerName="payload-pre-processing"
Oct 30 21:55:34.914714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.914605    5817 cpu_manager.go:414] "containerMap: removing container" podUID="99659e40-3d4e-42ad-83a6-310398106827" containerName="payload-processing"
Oct 30 21:55:34.914714 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.914611    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99659e40-3d4e-42ad-83a6-310398106827" containerName="payload-processing"
Oct 30 21:55:34.914783 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.914738    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="14a4a5ad-8af5-4a47-8b77-1ffa94872da0" containerName="payload-pre-processing"
Oct 30 21:55:34.914783 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.914746    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="99659e40-3d4e-42ad-83a6-310398106827" containerName="payload-processing"
Oct 30 21:55:34.920100 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.920077    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:34.993633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.993597    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85da0cc8-f192-4bfa-8a98-1a5740da53c1-openshift-service-ca\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:34.993785 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.993757    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:34.993838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.993823    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmkn7\" (UniqueName: \"kubernetes.io/projected/85da0cc8-f192-4bfa-8a98-1a5740da53c1-kube-api-access-rmkn7\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:34.993905 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:34.993893    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-metrics-certs\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.018899 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.018848    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2df7b6-serving-cert\""
Oct 30 21:55:35.024661 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.024630    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"]
Oct 30 21:55:35.095598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.095568    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85da0cc8-f192-4bfa-8a98-1a5740da53c1-openshift-service-ca\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.095751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.095640    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.095751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.095671    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmkn7\" (UniqueName: \"kubernetes.io/projected/85da0cc8-f192-4bfa-8a98-1a5740da53c1-kube-api-access-rmkn7\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.095751 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.095721    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-metrics-certs\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.095929 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:35.095782    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-serving-cert: secret "maas-api-e2e-api-b-2df7b6-serving-cert" not found
Oct 30 21:55:35.095929 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:35.095847    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls podName:85da0cc8-f192-4bfa-8a98-1a5740da53c1 nodeName:}" failed. No retries permitted until 2026-10-30 21:55:35.595831617 +0000 UTC m=+2470.883554161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls") pod "maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj" (UID: "85da0cc8-f192-4bfa-8a98-1a5740da53c1") : secret "maas-api-e2e-api-b-2df7b6-serving-cert" not found
Oct 30 21:55:35.096305 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.096281    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85da0cc8-f192-4bfa-8a98-1a5740da53c1-openshift-service-ca\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.098230 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.098206    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-metrics-certs\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.188287 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.188213    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmkn7\" (UniqueName: \"kubernetes.io/projected/85da0cc8-f192-4bfa-8a98-1a5740da53c1-kube-api-access-rmkn7\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.604144 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.604111    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.606569 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.606547    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls\") pod \"maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:35.834493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:35.834460    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:36.224649 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:55:36.224593    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85da0cc8_f192_4bfa_8a98_1a5740da53c1.slice/crio-85ae8d8d9e9f4c2ccd4e36237b380514619fac567f314374687cddea7a110551 WatchSource:0}: Error finding container 85ae8d8d9e9f4c2ccd4e36237b380514619fac567f314374687cddea7a110551: Status 404 returned error can't find the container with id 85ae8d8d9e9f4c2ccd4e36237b380514619fac567f314374687cddea7a110551
Oct 30 21:55:36.235668 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:36.235643    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"]
Oct 30 21:55:36.753621 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:36.753582    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj" event={"ID":"85da0cc8-f192-4bfa-8a98-1a5740da53c1","Type":"ContainerStarted","Data":"85ae8d8d9e9f4c2ccd4e36237b380514619fac567f314374687cddea7a110551"}
Oct 30 21:55:37.761859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:37.761823    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj" event={"ID":"85da0cc8-f192-4bfa-8a98-1a5740da53c1","Type":"ContainerStarted","Data":"d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30"}
Oct 30 21:55:37.762284 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:37.761919    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:37.876075 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:37.876025    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj" podStartSLOduration=3.041289746 podStartE2EDuration="3.876010756s" totalImagesPullingTime="834.72101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:37.8748779 +0000 UTC m=+2473.162600447" watchObservedRunningTime="2026-10-30 21:55:37.876010756 +0000 UTC m=+2473.163733302"
Oct 30 21:55:43.774173 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:43.774144    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 21:55:46.358801 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:55:46.358767    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:55:57.066122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.066075    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerID="429d30769691934013a89accc134839e359b067a75bd1d05be429df09aa78cc5" exitCode=0
Oct 30 21:55:57.066719 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.066155    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" event={"ID":"0a0ddf30-bcf7-4449-8871-9245159c0b09","Type":"ContainerDied","Data":"429d30769691934013a89accc134839e359b067a75bd1d05be429df09aa78cc5"}
Oct 30 21:55:57.066719 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.066627    5817 scope.go:122] "RemoveContainer" containerID="429d30769691934013a89accc134839e359b067a75bd1d05be429df09aa78cc5"
Oct 30 21:55:57.068379 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.068315    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:57.068474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.068404    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:57.068474 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.068435    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:57.723506 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.723460    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:55:57.723711 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:57.723527    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:55:58.081836 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:58.081756    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" event={"ID":"0a0ddf30-bcf7-4449-8871-9245159c0b09","Type":"ContainerStarted","Data":"ddad09c07560489410506faab96eda27b39ba978817e79dbdc322bfb87d7b694"}
Oct 30 21:55:58.304864 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:58.304774    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:55:58.309792 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:58.309416    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 21:55:58.309792 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:58.309491    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 21:55:59.115160 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.115128    5817 generic.go:356] "Generic (PLEG): container finished" podID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerID="8194d05fe70d6f5fd20e59c4858c739dc51af51b3b940f7ba613268e5f505e4f" exitCode=0
Oct 30 21:55:59.116262 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.116234    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" event={"ID":"4dcbc03b-703e-4e1c-a76c-339a0bc312a3","Type":"ContainerDied","Data":"8194d05fe70d6f5fd20e59c4858c739dc51af51b3b940f7ba613268e5f505e4f"}
Oct 30 21:55:59.116658 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.116637    5817 scope.go:122] "RemoveContainer" containerID="8194d05fe70d6f5fd20e59c4858c739dc51af51b3b940f7ba613268e5f505e4f"
Oct 30 21:55:59.119135 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.119094    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:59.119252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.119198    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:59.119312 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.119250    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:55:59.315375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.313739    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 21:55:59.315375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.313799    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 21:55:59.477176 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:55:59.477142    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:56:00.129839 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:00.129787    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" event={"ID":"4dcbc03b-703e-4e1c-a76c-339a0bc312a3","Type":"ContainerStarted","Data":"4b4ad4fec0a6587df0ec50647f36cba90527769b1ddc027124e189c38cd9f84b"}
Oct 30 21:56:00.161054 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:00.161010    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:56:00.161264 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:00.161244    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Oct 30 21:56:00.161320 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:00.161286    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Oct 30 21:56:00.327114 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:00.327074    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 21:56:00.327435 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:00.327390    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 21:56:01.160922 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:01.160874    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Oct 30 21:56:01.161419 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:01.160951    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Oct 30 21:56:01.296368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:01.293631    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 21:56:01.296368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:01.293699    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 21:56:02.160945 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:02.160915    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Oct 30 21:56:02.161337 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:02.160965    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Oct 30 21:56:02.296368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:02.292655    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 21:56:02.296368 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:02.292721    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 21:56:03.160953 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:03.160921    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Oct 30 21:56:03.161338 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:03.160974    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Oct 30 21:56:03.310370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:03.301663    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused" start-of-body=
Oct 30 21:56:03.310370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:03.301726    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.113:15021/healthz/ready\": dial tcp 10.134.0.113:15021: connect: connection refused"
Oct 30 21:56:04.164590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.161718    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Oct 30 21:56:04.164590 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.161784    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Oct 30 21:56:04.318370 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.316667    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:56:04.331373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.329586    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 21:56:04.331639 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.331614    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"]
Oct 30 21:56:04.335308 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.335286    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.370738 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.370712    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"]
Oct 30 21:56:04.433479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.432428    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-c4e874-dockercfg-nzkls\""
Oct 30 21:56:04.433479 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.433041    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-c4e874\""
Oct 30 21:56:04.447273 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.447220    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"]
Oct 30 21:56:04.473373 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.458621    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qr45\" (UniqueName: \"kubernetes.io/projected/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-kube-api-access-8qr45\") pod \"payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.578749 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.463182    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:04.641777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.584246    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.642471 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.642391    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"]
Oct 30 21:56:04.748903 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.748852    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qr45\" (UniqueName: \"kubernetes.io/projected/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-kube-api-access-8qr45\") pod \"payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.749081 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.748974    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87cqg\" (UniqueName: \"kubernetes.io/projected/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-kube-api-access-87cqg\") pod \"payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") " pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:04.749138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.749102    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.749178 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.749148    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") " pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:04.749788 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.749761    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.826512 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.826481    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qr45\" (UniqueName: \"kubernetes.io/projected/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-kube-api-access-8qr45\") pod \"payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.850067 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.850037    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87cqg\" (UniqueName: \"kubernetes.io/projected/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-kube-api-access-87cqg\") pod \"payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") " pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:04.850231 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.850115    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") " pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:04.850793 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.850767    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") " pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:04.916430 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.916375    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87cqg\" (UniqueName: \"kubernetes.io/projected/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-kube-api-access-87cqg\") pod \"payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") " pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:04.949507 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.949472    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:04.958838 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:04.958811    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:05.160894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:05.160568    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Oct 30 21:56:05.160894 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:05.160619    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Oct 30 21:56:05.368815 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:05.368781    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"]
Oct 30 21:56:05.418294 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:05.418268    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"]
Oct 30 21:56:05.421597 ip-10-0-142-25 kubenswrapper[5817]: W1030 21:56:05.421560    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ca11db_e5cd_4e70_b5d8_1a85e62eb0fd.slice/crio-648aa1f7cb0a75547527bc95e38bce032c3243dcb9cbb183188a3c08bbc8d67c WatchSource:0}: Error finding container 648aa1f7cb0a75547527bc95e38bce032c3243dcb9cbb183188a3c08bbc8d67c: Status 404 returned error can't find the container with id 648aa1f7cb0a75547527bc95e38bce032c3243dcb9cbb183188a3c08bbc8d67c
Oct 30 21:56:06.165293 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.165261    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:56:06.165763 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.165690    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:56:06.166468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.166445    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:56:06.230485 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.230455    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc" event={"ID":"9d7ba991-413e-4ade-a5e7-60ee2ef3446f","Type":"ContainerStarted","Data":"421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843"}
Oct 30 21:56:06.230623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.230498    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc" event={"ID":"9d7ba991-413e-4ade-a5e7-60ee2ef3446f","Type":"ContainerStarted","Data":"61866d70556ba70d9d9a3067003d5e2f360d420813d42295cc6e83e89697f970"}
Oct 30 21:56:06.230623 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.230524    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:06.232224 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.232196    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7" event={"ID":"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd","Type":"ContainerStarted","Data":"fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140"}
Oct 30 21:56:06.232364 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.232248    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7" event={"ID":"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd","Type":"ContainerStarted","Data":"648aa1f7cb0a75547527bc95e38bce032c3243dcb9cbb183188a3c08bbc8d67c"}
Oct 30 21:56:06.232446 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.232430    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:06.394104 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.394046    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7" podStartSLOduration=2.394031473 podStartE2EDuration="2.394031473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:06.391921792 +0000 UTC m=+2501.679644349" watchObservedRunningTime="2026-10-30 21:56:06.394031473 +0000 UTC m=+2501.681754038"
Oct 30 21:56:06.637300 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:06.637234    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc" podStartSLOduration=2.637219501 podStartE2EDuration="2.637219501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:06.635822158 +0000 UTC m=+2501.923544704" watchObservedRunningTime="2026-10-30 21:56:06.637219501 +0000 UTC m=+2501.924942046"
Oct 30 21:56:16.338554 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:56:16.338511    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:56:17.243181 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:17.243151    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 21:56:17.243407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:17.243308    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 21:56:37.011705 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:37.011661    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:56:37.012204 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:37.011721    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:38.484277 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:38.484242    5817 generic.go:356] "Generic (PLEG): container finished" podID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerID="fa612a3be8b390420799ca6d42c486dd996d1fe882ccfb3b7eaf73ffb7f6c1c7" exitCode=0
Oct 30 21:56:38.484707 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:38.484319    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" event={"ID":"d4f91b4f-7f3f-492a-aaea-d71076cde8c5","Type":"ContainerDied","Data":"fa612a3be8b390420799ca6d42c486dd996d1fe882ccfb3b7eaf73ffb7f6c1c7"}
Oct 30 21:56:38.484789 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:38.484773    5817 scope.go:122] "RemoveContainer" containerID="fa612a3be8b390420799ca6d42c486dd996d1fe882ccfb3b7eaf73ffb7f6c1c7"
Oct 30 21:56:38.486526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:38.486492    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:38.486663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:38.486580    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:38.486663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:38.486632    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 21:56:39.515722 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:39.515677    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" event={"ID":"d4f91b4f-7f3f-492a-aaea-d71076cde8c5","Type":"ContainerStarted","Data":"1d21dc51ed948166c592ae735d5e871ffa5835601abfa28cc8be361920c40bad"}
Oct 30 21:56:40.184493 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:40.184461    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:56:40.184771 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:40.184753    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:56:40.184818 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:40.184796    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 21:56:41.184748 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:41.184714    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:56:41.185166 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:41.184773    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 21:56:42.185374 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:42.185325    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:56:42.185773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:42.185396    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 21:56:43.184650 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:43.184612    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:56:43.184843 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:43.184676    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 21:56:44.184637 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:44.184605    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 21:56:44.185029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:44.184668    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 21:56:45.189473 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:45.189441    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:56:45.189948 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:45.189814    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:56:45.190428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:45.190405    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 21:56:46.339591 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:56:46.339550    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:56:59.292526 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:59.292461    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" probeResult="failure" output="Get \"https://10.134.0.65:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:59.632613 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:59.632524    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:56:59.728631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:56:59.728039    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:57:16.339505 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:57:16.339466    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:57:46.339800 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:57:46.339763    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:57:49.511088 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.511051    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"]
Oct 30 21:57:49.511733 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.511256    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7" podUID="a52eda1a-e83b-4967-9410-15b96a743607" containerName="maas-api" containerID="cri-o://41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283" gracePeriod=30
Oct 30 21:57:49.795334 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.795304    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:57:49.812257 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.812221    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-metrics-certs\") pod \"a52eda1a-e83b-4967-9410-15b96a743607\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") "
Oct 30 21:57:49.812438 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.812360    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a52eda1a-e83b-4967-9410-15b96a743607-kube-api-access-bvfqt\" (UniqueName: \"kubernetes.io/projected/a52eda1a-e83b-4967-9410-15b96a743607-kube-api-access-bvfqt\") pod \"a52eda1a-e83b-4967-9410-15b96a743607\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") "
Oct 30 21:57:49.812644 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.812628    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a52eda1a-e83b-4967-9410-15b96a743607-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52eda1a-e83b-4967-9410-15b96a743607-openshift-service-ca\") pod \"a52eda1a-e83b-4967-9410-15b96a743607\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") "
Oct 30 21:57:49.812732 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.812680    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls\") pod \"a52eda1a-e83b-4967-9410-15b96a743607\" (UID: \"a52eda1a-e83b-4967-9410-15b96a743607\") "
Oct 30 21:57:49.813036 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.813007    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52eda1a-e83b-4967-9410-15b96a743607-openshift-service-ca" pod "a52eda1a-e83b-4967-9410-15b96a743607" (UID: "a52eda1a-e83b-4967-9410-15b96a743607"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:49.813384 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.813336    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52eda1a-e83b-4967-9410-15b96a743607-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:49.814580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.814554    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52eda1a-e83b-4967-9410-15b96a743607-kube-api-access-bvfqt" pod "a52eda1a-e83b-4967-9410-15b96a743607" (UID: "a52eda1a-e83b-4967-9410-15b96a743607"). InnerVolumeSpecName "kube-api-access-bvfqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:49.814665 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.814579    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-metrics-certs" pod "a52eda1a-e83b-4967-9410-15b96a743607" (UID: "a52eda1a-e83b-4967-9410-15b96a743607"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:49.815020 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.814704    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls" pod "a52eda1a-e83b-4967-9410-15b96a743607" (UID: "a52eda1a-e83b-4967-9410-15b96a743607"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:49.916730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.916600    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:49.916730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.916650    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52eda1a-e83b-4967-9410-15b96a743607-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:49.916730 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:49.916672    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvfqt\" (UniqueName: \"kubernetes.io/projected/a52eda1a-e83b-4967-9410-15b96a743607-kube-api-access-bvfqt\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:50.290853 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.290816    5817 generic.go:356] "Generic (PLEG): container finished" podID="a52eda1a-e83b-4967-9410-15b96a743607" containerID="41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283" exitCode=0
Oct 30 21:57:50.291029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.290859    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"
Oct 30 21:57:50.291029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.290900    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7" event={"ID":"a52eda1a-e83b-4967-9410-15b96a743607","Type":"ContainerDied","Data":"41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283"}
Oct 30 21:57:50.291029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.290954    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7" event={"ID":"a52eda1a-e83b-4967-9410-15b96a743607","Type":"ContainerDied","Data":"a7df7b9a3f6b227dd8e99b9a3ed1885da45b376fec25250369ba46704b14d838"}
Oct 30 21:57:50.291029 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.290979    5817 scope.go:122] "RemoveContainer" containerID="41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283"
Oct 30 21:57:50.302383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.302362    5817 scope.go:122] "RemoveContainer" containerID="41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283"
Oct 30 21:57:50.302640 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:57:50.302621    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283\": container with ID starting with 41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283 not found: ID does not exist" containerID="41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283"
Oct 30 21:57:50.302700 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.302649    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283"} err="failed to get container status \"41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283\": rpc error: code = NotFound desc = could not find container \"41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283\": container with ID starting with 41632de10bcecf6519150a6ab755e460503c50bf19a134f16147cad174a9b283 not found: ID does not exist"
Oct 30 21:57:50.475445 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.475400    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"]
Oct 30 21:57:50.536395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.527183    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bfdf30b9-7d4557498b-zkjm7"]
Oct 30 21:57:50.845422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:57:50.845393    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a52eda1a-e83b-4967-9410-15b96a743607" path="/var/lib/kubelet/pods/a52eda1a-e83b-4967-9410-15b96a743607/volumes"
Oct 30 21:58:16.340505 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:58:16.340423    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:58:20.345736 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.345682    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"]
Oct 30 21:58:20.346203 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.346015    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25" podUID="b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" containerName="maas-api" containerID="cri-o://390ad2d68a08668a110c297509b62f7c7203f8cd662211f751c935a3230ab7ba" gracePeriod=30
Oct 30 21:58:20.523215 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.523181    5817 generic.go:356] "Generic (PLEG): container finished" podID="b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" containerID="390ad2d68a08668a110c297509b62f7c7203f8cd662211f751c935a3230ab7ba" exitCode=0
Oct 30 21:58:20.523407 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.523258    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25" event={"ID":"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9","Type":"ContainerDied","Data":"390ad2d68a08668a110c297509b62f7c7203f8cd662211f751c935a3230ab7ba"}
Oct 30 21:58:20.672797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.672772    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:58:20.814375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.814009    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-metrics-certs\") pod \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") "
Oct 30 21:58:20.814375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.814120    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-openshift-service-ca\") pod \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") "
Oct 30 21:58:20.814375 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.814189    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls\") pod \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") "
Oct 30 21:58:20.814700 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.814429    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-kube-api-access-wrls5\" (UniqueName: \"kubernetes.io/projected/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-kube-api-access-wrls5\") pod \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\" (UID: \"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9\") "
Oct 30 21:58:20.815140 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.815091    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-openshift-service-ca" pod "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" (UID: "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:20.815403 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.815319    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:20.817950 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.816834    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-metrics-certs" pod "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" (UID: "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:20.821429 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.818372    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls" pod "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" (UID: "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:20.822897 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.822838    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-kube-api-access-wrls5" pod "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" (UID: "b7cad3a9-6764-4474-9e26-d8e5b9abc0e9"). InnerVolumeSpecName "kube-api-access-wrls5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:20.917209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.917124    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrls5\" (UniqueName: \"kubernetes.io/projected/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-kube-api-access-wrls5\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:20.917209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.917160    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:20.917209 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:20.917170    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:21.532185 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:21.532154    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"
Oct 30 21:58:21.532592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:21.532184    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25" event={"ID":"b7cad3a9-6764-4474-9e26-d8e5b9abc0e9","Type":"ContainerDied","Data":"bfc6248a8e2dfb5fd81bb6e3848bce685feaa6c95999091167528c2ce31d6b46"}
Oct 30 21:58:21.532592 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:21.532281    5817 scope.go:122] "RemoveContainer" containerID="390ad2d68a08668a110c297509b62f7c7203f8cd662211f751c935a3230ab7ba"
Oct 30 21:58:21.689317 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:21.689284    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"]
Oct 30 21:58:21.736195 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:21.736167    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-7d7f57-745675fd6b-mqd25"]
Oct 30 21:58:22.846041 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:22.846002    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" path="/var/lib/kubelet/pods/b7cad3a9-6764-4474-9e26-d8e5b9abc0e9/volumes"
Oct 30 21:58:31.522379 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:31.522335    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"]
Oct 30 21:58:31.522791 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:31.522576    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" containerID="cri-o://4b4ad4fec0a6587df0ec50647f36cba90527769b1ddc027124e189c38cd9f84b" gracePeriod=30
Oct 30 21:58:36.167876 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.167840    5817 patch_prober.go:29] interesting pod/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:58:36.168285 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.167903    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:58:36.729174 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.729143    5817 generic.go:356] "Generic (PLEG): container finished" podID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerID="4b4ad4fec0a6587df0ec50647f36cba90527769b1ddc027124e189c38cd9f84b" exitCode=0
Oct 30 21:58:36.729328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.729223    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" event={"ID":"4dcbc03b-703e-4e1c-a76c-339a0bc312a3","Type":"ContainerDied","Data":"4b4ad4fec0a6587df0ec50647f36cba90527769b1ddc027124e189c38cd9f84b"}
Oct 30 21:58:36.729328 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.729301    5817 scope.go:122] "RemoveContainer" containerID="8194d05fe70d6f5fd20e59c4858c739dc51af51b3b940f7ba613268e5f505e4f"
Oct 30 21:58:36.804632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.804576    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:58:36.905046 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905018    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-envoy\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905192 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905060    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-socket\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905284 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905266    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-kube-api-access-jss99\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-kube-api-access-jss99\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905326 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905306    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istiod-ca-cert\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905323    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-socket" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:36.905396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905361    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-data\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905389    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-podinfo\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905430    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-certs\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905474    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-token\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-token\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905545 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905513    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-credential-socket\") pod \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\" (UID: \"4dcbc03b-703e-4e1c-a76c-339a0bc312a3\") "
Oct 30 21:58:36.905736 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905680    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-data" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:36.905807 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905753    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-certs" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:36.905807 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905788    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istiod-ca-cert" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:36.905971 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.905950    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-credential-socket" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:36.906275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.906252    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:36.906378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.906281    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:36.906378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.906297    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:36.906378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.906313    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:36.906378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.906329    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:36.907404 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.907378    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-envoy" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:36.907720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.907697    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-podinfo" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:58:36.907813 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.907768    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-token" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:36.907813 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:36.907802    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-kube-api-access-jss99" pod "4dcbc03b-703e-4e1c-a76c-339a0bc312a3" (UID: "4dcbc03b-703e-4e1c-a76c-339a0bc312a3"). InnerVolumeSpecName "kube-api-access-jss99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:37.008267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:37.008239    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:37.008267 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:37.008267    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:37.008468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:37.008301    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jss99\" (UniqueName: \"kubernetes.io/projected/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-kube-api-access-jss99\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:37.008468 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:37.008327    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dcbc03b-703e-4e1c-a76c-339a0bc312a3-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:37.741846 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:37.741814    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"
Oct 30 21:58:37.742378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:37.741876    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs" event={"ID":"4dcbc03b-703e-4e1c-a76c-339a0bc312a3","Type":"ContainerDied","Data":"78468b142c2dd0f2ddb1f016b2e1dcfec5c2656b2d458706a41b71a2839e595f"}
Oct 30 21:58:37.742378 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:37.741922    5817 scope.go:122] "RemoveContainer" containerID="4b4ad4fec0a6587df0ec50647f36cba90527769b1ddc027124e189c38cd9f84b"
Oct 30 21:58:38.019632 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:38.019500    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"]
Oct 30 21:58:38.042653 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:38.042609    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-7d7f57-openshift-default-6c88898775-x8bhs"]
Oct 30 21:58:38.364122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:38.364045    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"]
Oct 30 21:58:38.849068 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:38.849034    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" path="/var/lib/kubelet/pods/4dcbc03b-703e-4e1c-a76c-339a0bc312a3/volumes"
Oct 30 21:58:45.507439 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:45.507396    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"]
Oct 30 21:58:45.619584 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:45.619545    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"]
Oct 30 21:58:46.339179 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:58:46.339140    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:58:50.508598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:50.508520    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" podUID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" containerName="payload-processing" containerID="cri-o://98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf" gracePeriod=25
Oct 30 21:58:50.620797 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:50.620752    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" podUID="dfb8c540-571e-4f74-99c0-a6dc852fe900" containerName="payload-pre-processing" containerID="cri-o://874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64" gracePeriod=25
Oct 30 21:58:53.406696 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.406634    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main" containerID="cri-o://de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8" gracePeriod=45
Oct 30 21:58:53.691517 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.691488    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:58:53.875870 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.875826    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-model-cache\") pod \"865b13f8-87cb-496f-b90f-080ed2aaff6e\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") "
Oct 30 21:58:53.876257 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.876113    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-model-cache" pod "865b13f8-87cb-496f-b90f-080ed2aaff6e" (UID: "865b13f8-87cb-496f-b90f-080ed2aaff6e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:53.876257 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.876234    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-home\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-home\") pod \"865b13f8-87cb-496f-b90f-080ed2aaff6e\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") "
Oct 30 21:58:53.876504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.876280    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-tmp-dir\") pod \"865b13f8-87cb-496f-b90f-080ed2aaff6e\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") "
Oct 30 21:58:53.876504 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.876328    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-dshm\") pod \"865b13f8-87cb-496f-b90f-080ed2aaff6e\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") "
Oct 30 21:58:53.876619 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.876523    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/865b13f8-87cb-496f-b90f-080ed2aaff6e-kube-api-access-hdk4k\" (UniqueName: \"kubernetes.io/projected/865b13f8-87cb-496f-b90f-080ed2aaff6e-kube-api-access-hdk4k\") pod \"865b13f8-87cb-496f-b90f-080ed2aaff6e\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") "
Oct 30 21:58:53.876677 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.876625    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/865b13f8-87cb-496f-b90f-080ed2aaff6e-tls-certs\" (UniqueName: \"kubernetes.io/secret/865b13f8-87cb-496f-b90f-080ed2aaff6e-tls-certs\") pod \"865b13f8-87cb-496f-b90f-080ed2aaff6e\" (UID: \"865b13f8-87cb-496f-b90f-080ed2aaff6e\") "
Oct 30 21:58:53.877723 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.877674    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-home" pod "865b13f8-87cb-496f-b90f-080ed2aaff6e" (UID: "865b13f8-87cb-496f-b90f-080ed2aaff6e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:53.877983 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.877935    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-tmp-dir" pod "865b13f8-87cb-496f-b90f-080ed2aaff6e" (UID: "865b13f8-87cb-496f-b90f-080ed2aaff6e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:53.905395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.880207    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865b13f8-87cb-496f-b90f-080ed2aaff6e-tls-certs" pod "865b13f8-87cb-496f-b90f-080ed2aaff6e" (UID: "865b13f8-87cb-496f-b90f-080ed2aaff6e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:53.905395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.880940    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865b13f8-87cb-496f-b90f-080ed2aaff6e-kube-api-access-hdk4k" pod "865b13f8-87cb-496f-b90f-080ed2aaff6e" (UID: "865b13f8-87cb-496f-b90f-080ed2aaff6e"). InnerVolumeSpecName "kube-api-access-hdk4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:53.905395 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.881107    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-dshm" pod "865b13f8-87cb-496f-b90f-080ed2aaff6e" (UID: "865b13f8-87cb-496f-b90f-080ed2aaff6e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:53.929538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.929465    5817 generic.go:356] "Generic (PLEG): container finished" podID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerID="de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8" exitCode=0
Oct 30 21:58:53.929538 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.929525    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"
Oct 30 21:58:53.929720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.929551    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" event={"ID":"865b13f8-87cb-496f-b90f-080ed2aaff6e","Type":"ContainerDied","Data":"de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8"}
Oct 30 21:58:53.929720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.929615    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j" event={"ID":"865b13f8-87cb-496f-b90f-080ed2aaff6e","Type":"ContainerDied","Data":"a896a4e05fdf3e25eb9342506aa80526f1cfd086c72da416246b7c6376b771cd"}
Oct 30 21:58:53.929720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.929639    5817 scope.go:122] "RemoveContainer" containerID="de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8"
Oct 30 21:58:53.949469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.949448    5817 scope.go:122] "RemoveContainer" containerID="de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8"
Oct 30 21:58:53.949775 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:58:53.949754    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8\": container with ID starting with de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8 not found: ID does not exist" containerID="de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8"
Oct 30 21:58:53.949845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.949786    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8"} err="failed to get container status \"de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8\": rpc error: code = NotFound desc = could not find container \"de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8\": container with ID starting with de62c2544b40c6fc60bf6073df111b17749055b5b6678227d042d1860950b4e8 not found: ID does not exist"
Oct 30 21:58:53.979145 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.979119    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdk4k\" (UniqueName: \"kubernetes.io/projected/865b13f8-87cb-496f-b90f-080ed2aaff6e-kube-api-access-hdk4k\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:53.979145 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.979149    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b13f8-87cb-496f-b90f-080ed2aaff6e-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:53.979290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.979167    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:53.979290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.979182    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:53.979290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.979198    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:53.979290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:53.979212    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b13f8-87cb-496f-b90f-080ed2aaff6e-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:54.104304 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:54.104259    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"]
Oct 30 21:58:54.120106 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:54.120073    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-554f46/distinct-w4-554f46-kserve-795bf7c4c9-jgz7j"]
Oct 30 21:58:54.864886 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:54.853103    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" path="/var/lib/kubelet/pods/865b13f8-87cb-496f-b90f-080ed2aaff6e/volumes"
Oct 30 21:58:56.477372 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:56.476462    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" podUID="dfb8c540-571e-4f74-99c0-a6dc852fe900" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:58:58.470655 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:58:58.470614    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" podUID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:59:06.456276 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:06.456232    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" podUID="dfb8c540-571e-4f74-99c0-a6dc852fe900" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:59:08.470503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:08.470463    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" podUID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:59:10.861482 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:10.861453    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:59:10.916571 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:10.916544    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:59:10.941315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:10.941286    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f49b2817-d5d6-438f-819a-58a8fbc69e3d-kube-api-access-np8wf\" (UniqueName: \"kubernetes.io/projected/f49b2817-d5d6-438f-819a-58a8fbc69e3d-kube-api-access-np8wf\") pod \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") "
Oct 30 21:59:10.941466 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:10.941321    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f49b2817-d5d6-438f-819a-58a8fbc69e3d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49b2817-d5d6-438f-819a-58a8fbc69e3d-plugins-config-volume\") pod \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\" (UID: \"f49b2817-d5d6-438f-819a-58a8fbc69e3d\") "
Oct 30 21:59:10.941695 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:10.941673    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f49b2817-d5d6-438f-819a-58a8fbc69e3d-plugins-config-volume" pod "f49b2817-d5d6-438f-819a-58a8fbc69e3d" (UID: "f49b2817-d5d6-438f-819a-58a8fbc69e3d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:10.942128 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:10.942106    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49b2817-d5d6-438f-819a-58a8fbc69e3d-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:10.943340 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:10.943323    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49b2817-d5d6-438f-819a-58a8fbc69e3d-kube-api-access-np8wf" pod "f49b2817-d5d6-438f-819a-58a8fbc69e3d" (UID: "f49b2817-d5d6-438f-819a-58a8fbc69e3d"). InnerVolumeSpecName "kube-api-access-np8wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:11.043704 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.043674    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfb8c540-571e-4f74-99c0-a6dc852fe900-kube-api-access-grlns\" (UniqueName: \"kubernetes.io/projected/dfb8c540-571e-4f74-99c0-a6dc852fe900-kube-api-access-grlns\") pod \"dfb8c540-571e-4f74-99c0-a6dc852fe900\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") "
Oct 30 21:59:11.043858 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.043830    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfb8c540-571e-4f74-99c0-a6dc852fe900-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfb8c540-571e-4f74-99c0-a6dc852fe900-plugins-config-volume\") pod \"dfb8c540-571e-4f74-99c0-a6dc852fe900\" (UID: \"dfb8c540-571e-4f74-99c0-a6dc852fe900\") "
Oct 30 21:59:11.044199 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.044178    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfb8c540-571e-4f74-99c0-a6dc852fe900-plugins-config-volume" pod "dfb8c540-571e-4f74-99c0-a6dc852fe900" (UID: "dfb8c540-571e-4f74-99c0-a6dc852fe900"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:11.044472 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.044458    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np8wf\" (UniqueName: \"kubernetes.io/projected/f49b2817-d5d6-438f-819a-58a8fbc69e3d-kube-api-access-np8wf\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:11.044532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.044476    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfb8c540-571e-4f74-99c0-a6dc852fe900-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:11.045939 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.045914    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb8c540-571e-4f74-99c0-a6dc852fe900-kube-api-access-grlns" pod "dfb8c540-571e-4f74-99c0-a6dc852fe900" (UID: "dfb8c540-571e-4f74-99c0-a6dc852fe900"). InnerVolumeSpecName "kube-api-access-grlns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:11.140102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.140002    5817 generic.go:356] "Generic (PLEG): container finished" podID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" containerID="98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf" exitCode=0
Oct 30 21:59:11.140102 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.140047    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" event={"ID":"f49b2817-d5d6-438f-819a-58a8fbc69e3d","Type":"ContainerDied","Data":"98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf"}
Oct 30 21:59:11.140319 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.140122    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx" event={"ID":"f49b2817-d5d6-438f-819a-58a8fbc69e3d","Type":"ContainerDied","Data":"3231fd55f4090a4852f4185c895d6f7c3183837d8980177b75d229d8dd23df76"}
Oct 30 21:59:11.140319 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.140061    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"
Oct 30 21:59:11.140319 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.140153    5817 scope.go:122] "RemoveContainer" containerID="98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf"
Oct 30 21:59:11.141766 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.141743    5817 generic.go:356] "Generic (PLEG): container finished" podID="dfb8c540-571e-4f74-99c0-a6dc852fe900" containerID="874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64" exitCode=0
Oct 30 21:59:11.141899 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.141883    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" event={"ID":"dfb8c540-571e-4f74-99c0-a6dc852fe900","Type":"ContainerDied","Data":"874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64"}
Oct 30 21:59:11.141970 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.141928    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t" event={"ID":"dfb8c540-571e-4f74-99c0-a6dc852fe900","Type":"ContainerDied","Data":"f286686faccbec9eb0c3a86b99f41e5702cb9f44283d435c41e566d7cbdcc40d"}
Oct 30 21:59:11.142030 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.142002    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"
Oct 30 21:59:11.145692 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.145666    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grlns\" (UniqueName: \"kubernetes.io/projected/dfb8c540-571e-4f74-99c0-a6dc852fe900-kube-api-access-grlns\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:11.152387 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.152337    5817 scope.go:122] "RemoveContainer" containerID="98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf"
Oct 30 21:59:11.152663 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:11.152645    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf\": container with ID starting with 98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf not found: ID does not exist" containerID="98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf"
Oct 30 21:59:11.152712 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.152671    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf"} err="failed to get container status \"98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf\": rpc error: code = NotFound desc = could not find container \"98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf\": container with ID starting with 98faa9597ead793055a9dc9e25098f8c095ed695ab6f62acb687d24e091192cf not found: ID does not exist"
Oct 30 21:59:11.152712 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.152687    5817 scope.go:122] "RemoveContainer" containerID="874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64"
Oct 30 21:59:11.164210 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.164192    5817 scope.go:122] "RemoveContainer" containerID="874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64"
Oct 30 21:59:11.164521 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:11.164499    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64\": container with ID starting with 874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64 not found: ID does not exist" containerID="874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64"
Oct 30 21:59:11.164628 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.164529    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64"} err="failed to get container status \"874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64\": rpc error: code = NotFound desc = could not find container \"874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64\": container with ID starting with 874058a9095a1d5ec0389ad9357e023608662b6d71638b2affeeea79e50baa64 not found: ID does not exist"
Oct 30 21:59:11.306986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.306952    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"]
Oct 30 21:59:11.312335 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.312312    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-554f46-7f4cdd7fb6-57nwx"]
Oct 30 21:59:11.413397 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.413326    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"]
Oct 30 21:59:11.420282 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:11.420257    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-554f46-668c596d7-8cn8t"]
Oct 30 21:59:12.845040 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:12.845008    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfb8c540-571e-4f74-99c0-a6dc852fe900" path="/var/lib/kubelet/pods/dfb8c540-571e-4f74-99c0-a6dc852fe900/volumes"
Oct 30 21:59:12.845464 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:12.845449    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" path="/var/lib/kubelet/pods/f49b2817-d5d6-438f-819a-58a8fbc69e3d/volumes"
Oct 30 21:59:16.339747 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:16.339706    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:59:24.548845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:24.548809    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"]
Oct 30 21:59:24.549760 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:24.549025    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy" containerID="cri-o://e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d" gracePeriod=30
Oct 30 21:59:25.479428 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:25.479392    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"]
Oct 30 21:59:25.479663 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:25.479633    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" containerID="cri-o://a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51" gracePeriod=30
Oct 30 21:59:25.830559 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:25.830483    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:59:25.830559 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:25.830547    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:59:29.844161 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.844132    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:59:29.952427 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952391    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-podinfo\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.952598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952464    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-kube-api-access-jq2f6\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-kube-api-access-jq2f6\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.952598 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952553    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-socket\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.952720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952618    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-certs\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.952720 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952651    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-data\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.952829 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952722    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-token\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-token\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.952829 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952824    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f17ad69-4769-4a86-84da-5c3f9f2df240-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f17ad69-4769-4a86-84da-5c3f9f2df240-istiod-ca-cert\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.952940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952863    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-certs" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:29.952940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952875    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-socket" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:29.952940 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.952897    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-envoy\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.953105 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953018    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-credential-socket\") pod \"4f17ad69-4769-4a86-84da-5c3f9f2df240\" (UID: \"4f17ad69-4769-4a86-84da-5c3f9f2df240\") "
Oct 30 21:59:29.953188 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953168    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-data" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:29.953272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953185    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f17ad69-4769-4a86-84da-5c3f9f2df240-istiod-ca-cert" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:29.953499 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953477    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-credential-socket" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:29.953835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953813    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:29.953835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953840    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:29.953999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953857    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:29.953999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953872    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:29.953999 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.953888    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f17ad69-4769-4a86-84da-5c3f9f2df240-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:29.955114 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.955088    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-envoy" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:29.955114 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.955102    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-kube-api-access-jq2f6" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "kube-api-access-jq2f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:29.955281 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.955253    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-podinfo" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:59:29.955396 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:29.955380    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-token" pod "4f17ad69-4769-4a86-84da-5c3f9f2df240" (UID: "4f17ad69-4769-4a86-84da-5c3f9f2df240"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:30.055419 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.055342    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.055556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.055425    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.055556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.055440    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f17ad69-4769-4a86-84da-5c3f9f2df240-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.055556 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.055463    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq2f6\" (UniqueName: \"kubernetes.io/projected/4f17ad69-4769-4a86-84da-5c3f9f2df240-kube-api-access-jq2f6\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.313691 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.313662    5817 generic.go:356] "Generic (PLEG): container finished" podID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerID="e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d" exitCode=0
Oct 30 21:59:30.313866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.313708    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"
Oct 30 21:59:30.313866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.313749    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" event={"ID":"4f17ad69-4769-4a86-84da-5c3f9f2df240","Type":"ContainerDied","Data":"e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d"}
Oct 30 21:59:30.313866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.313806    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s" event={"ID":"4f17ad69-4769-4a86-84da-5c3f9f2df240","Type":"ContainerDied","Data":"76546e8936df4854307eca0cb4771081c2b4646ff7593528770dd9ca277891e1"}
Oct 30 21:59:30.313866 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.313831    5817 scope.go:122] "RemoveContainer" containerID="e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d"
Oct 30 21:59:30.325445 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.325250    5817 scope.go:122] "RemoveContainer" containerID="e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d"
Oct 30 21:59:30.325582 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:30.325561    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d\": container with ID starting with e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d not found: ID does not exist" containerID="e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d"
Oct 30 21:59:30.325645 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.325593    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d"} err="failed to get container status \"e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d\": rpc error: code = NotFound desc = could not find container \"e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d\": container with ID starting with e1385775fd84304a62a2e2402a2598bd5c5c7aa946c1846622af065b3ddccc7d not found: ID does not exist"
Oct 30 21:59:30.494047 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.494010    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"]
Oct 30 21:59:30.514422 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.514397    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-19dd4f-openshift-default-69f45f44bc-vxl7s"]
Oct 30 21:59:30.760761 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.760735    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:59:30.844641 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.844563    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" path="/var/lib/kubelet/pods/4f17ad69-4769-4a86-84da-5c3f9f2df240/volumes"
Oct 30 21:59:30.871703 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.871676    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-socket\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.871845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.871713    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/544e7dca-6988-491c-907b-e903dafe5c65-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544e7dca-6988-491c-907b-e903dafe5c65-istiod-ca-cert\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.871845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.871746    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-data\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.871845 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.871814    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/544e7dca-6988-491c-907b-e903dafe5c65-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544e7dca-6988-491c-907b-e903dafe5c65-istio-podinfo\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.872016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.871921    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-istio-token\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-istio-token\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.872016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.871938    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-socket" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:30.872016 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.871992    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-data" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:30.872188 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872079    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-envoy\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.872188 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872126    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-kube-api-access-28fsf\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-kube-api-access-28fsf\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.872188 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872165    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-credential-socket\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.872339 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872180    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/544e7dca-6988-491c-907b-e903dafe5c65-istiod-ca-cert" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:30.872339 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872212    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-certs\") pod \"544e7dca-6988-491c-907b-e903dafe5c65\" (UID: \"544e7dca-6988-491c-907b-e903dafe5c65\") "
Oct 30 21:59:30.872503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872472    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-credential-socket" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:30.872561 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872515    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-certs" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:30.872927 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872902    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.873013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872930    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544e7dca-6988-491c-907b-e903dafe5c65-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.873013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872946    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.873013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872963    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.873013 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.872977    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.874275 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.874252    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-envoy" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:30.874413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.874287    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-kube-api-access-28fsf" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "kube-api-access-28fsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:30.874413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.874295    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/544e7dca-6988-491c-907b-e903dafe5c65-istio-podinfo" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:59:30.874413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.874333    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-istio-token" pod "544e7dca-6988-491c-907b-e903dafe5c65" (UID: "544e7dca-6988-491c-907b-e903dafe5c65"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:30.974588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.974556    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.974588 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.974591    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544e7dca-6988-491c-907b-e903dafe5c65-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.974785 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.974610    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28fsf\" (UniqueName: \"kubernetes.io/projected/544e7dca-6988-491c-907b-e903dafe5c65-kube-api-access-28fsf\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:30.974785 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:30.974627    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544e7dca-6988-491c-907b-e903dafe5c65-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:31.321636 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.321605    5817 generic.go:356] "Generic (PLEG): container finished" podID="544e7dca-6988-491c-907b-e903dafe5c65" containerID="a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51" exitCode=0
Oct 30 21:59:31.321823 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.321649    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"
Oct 30 21:59:31.321823 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.321691    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" event={"ID":"544e7dca-6988-491c-907b-e903dafe5c65","Type":"ContainerDied","Data":"a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51"}
Oct 30 21:59:31.321823 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.321755    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d" event={"ID":"544e7dca-6988-491c-907b-e903dafe5c65","Type":"ContainerDied","Data":"04cc2165cc98aae2beaed988773bb6a7e5a28ffb0bcb835ac19b1a0badc6cdea"}
Oct 30 21:59:31.321823 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.321791    5817 scope.go:122] "RemoveContainer" containerID="a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51"
Oct 30 21:59:31.333138 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.333122    5817 scope.go:122] "RemoveContainer" containerID="a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51"
Oct 30 21:59:31.333407 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:31.333388    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51\": container with ID starting with a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51 not found: ID does not exist" containerID="a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51"
Oct 30 21:59:31.333457 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.333416    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51"} err="failed to get container status \"a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51\": rpc error: code = NotFound desc = could not find container \"a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51\": container with ID starting with a2345516ddfcfea3369dcbee77765c9ad6b5f98bad8b493650360b004599ba51 not found: ID does not exist"
Oct 30 21:59:31.483885 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.483831    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"]
Oct 30 21:59:31.492218 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:31.492193    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-9v94d"]
Oct 30 21:59:32.848257 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:32.848166    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="544e7dca-6988-491c-907b-e903dafe5c65" path="/var/lib/kubelet/pods/544e7dca-6988-491c-907b-e903dafe5c65/volumes"
Oct 30 21:59:32.877146 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:32.877110    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"]
Oct 30 21:59:32.877409 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:32.877376    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy" containerID="cri-o://da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633" gracePeriod=30
Oct 30 21:59:38.228955 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.228928    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:59:38.325874 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.325836    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-kube-api-access-nbjsc\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-kube-api-access-nbjsc\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326065 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.325970    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-certs\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326065 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326003    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-socket\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326065 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326024    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-credential-socket\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326077    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-token\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-token\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326141    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istiod-ca-cert\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326252 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326193    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-podinfo\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326303    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-envoy\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326318    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-socket" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:38.326424 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326330    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-certs" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:38.326548 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326442    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-credential-socket" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:38.326548 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326445    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-data\") pod \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\" (UID: \"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d\") "
Oct 30 21:59:38.326683 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326647    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-data" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:38.326683 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.326673    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istiod-ca-cert" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:38.327327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.327205    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.327327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.327233    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.327327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.327247    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.327327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.327261    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.327327 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.327274    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.328648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.328622    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-podinfo" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:59:38.328648 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.328632    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-envoy" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:38.328777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.328704    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-token" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:38.328777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.328721    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-kube-api-access-nbjsc" pod "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" (UID: "19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d"). InnerVolumeSpecName "kube-api-access-nbjsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:38.379337 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.379312    5817 generic.go:356] "Generic (PLEG): container finished" podID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerID="da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633" exitCode=0
Oct 30 21:59:38.379465 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.379379    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"
Oct 30 21:59:38.379465 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.379403    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" event={"ID":"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d","Type":"ContainerDied","Data":"da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633"}
Oct 30 21:59:38.379596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.379475    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp" event={"ID":"19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d","Type":"ContainerDied","Data":"491bb01e65f2e8f6c88c503e3843c7f69193654ee912a96326c929390a59bf3d"}
Oct 30 21:59:38.379596 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.379513    5817 scope.go:122] "RemoveContainer" containerID="da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633"
Oct 30 21:59:38.416537 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.416513    5817 scope.go:122] "RemoveContainer" containerID="3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae"
Oct 30 21:59:38.431806 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.431698    5817 scope.go:122] "RemoveContainer" containerID="da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633"
Oct 30 21:59:38.432162 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:38.432127    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633\": container with ID starting with da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633 not found: ID does not exist" containerID="da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633"
Oct 30 21:59:38.432272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.432171    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633"} err="failed to get container status \"da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633\": rpc error: code = NotFound desc = could not find container \"da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633\": container with ID starting with da91403393e42bde82062674f469d91cf9bbc7e9c561d9f72f086981eeb0d633 not found: ID does not exist"
Oct 30 21:59:38.432272 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.432202    5817 scope.go:122] "RemoveContainer" containerID="3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae"
Oct 30 21:59:38.432849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.432563    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.432849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.432589    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.432849 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:38.432588    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae\": container with ID starting with 3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae not found: ID does not exist" containerID="3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae"
Oct 30 21:59:38.432849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.432625    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae"} err="failed to get container status \"3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae\": rpc error: code = NotFound desc = could not find container \"3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae\": container with ID starting with 3d4ec8fa23301658134c6b85c19b1c9afea3d16494f609238ee9a0dee0f6a4ae not found: ID does not exist"
Oct 30 21:59:38.432849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.432605    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.432849 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.432668    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbjsc\" (UniqueName: \"kubernetes.io/projected/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d-kube-api-access-nbjsc\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:38.526122 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.525622    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"]
Oct 30 21:59:38.531570 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.531541    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-27114d-openshift-default-64d867bb7f-vzjxp"]
Oct 30 21:59:38.853935 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:38.853897    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" path="/var/lib/kubelet/pods/19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d/volumes"
Oct 30 21:59:39.062580 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:39.062544    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"]
Oct 30 21:59:39.062883 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:39.062849    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy" containerID="cri-o://216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0" gracePeriod=30
Oct 30 21:59:39.464969 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:39.464925    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"]
Oct 30 21:59:39.465503 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:39.465194    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" containerID="cri-o://e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2" gracePeriod=30
Oct 30 21:59:39.760221 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:39.760132    5817 patch_prober.go:29] interesting pod/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:59:39.760631 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:39.760588    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:59:44.348635 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.348559    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:59:44.447492 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.447462    5817 generic.go:356] "Generic (PLEG): container finished" podID="df235d02-6ff1-4fe3-8066-83027239875a" containerID="216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0" exitCode=0
Oct 30 21:59:44.447670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.447507    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"
Oct 30 21:59:44.447670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.447523    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" event={"ID":"df235d02-6ff1-4fe3-8066-83027239875a","Type":"ContainerDied","Data":"216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0"}
Oct 30 21:59:44.447670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.447586    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th" event={"ID":"df235d02-6ff1-4fe3-8066-83027239875a","Type":"ContainerDied","Data":"2f261b261dce5f19705a02f345c75e758df40531e4cdcdfea89ac80e5d6c6150"}
Oct 30 21:59:44.447670 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.447607    5817 scope.go:122] "RemoveContainer" containerID="216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0"
Oct 30 21:59:44.455038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455012    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-credential-socket\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455154 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455044    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-socket\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455154 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455093    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-data\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455154    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-istio-token\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-istio-token\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455215    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df235d02-6ff1-4fe3-8066-83027239875a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df235d02-6ff1-4fe3-8066-83027239875a-istiod-ca-cert\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455259    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df235d02-6ff1-4fe3-8066-83027239875a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df235d02-6ff1-4fe3-8066-83027239875a-istio-podinfo\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455456 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455279    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-credential-socket" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.455456 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455414    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-data" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.455456 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455428    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-certs\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455457    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-socket" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.455608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455473    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-kube-api-access-sqxfz\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-kube-api-access-sqxfz\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455608 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455588    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-envoy\") pod \"df235d02-6ff1-4fe3-8066-83027239875a\" (UID: \"df235d02-6ff1-4fe3-8066-83027239875a\") "
Oct 30 21:59:44.455778 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455686    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-certs" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.455778 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.455698    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df235d02-6ff1-4fe3-8066-83027239875a-istiod-ca-cert" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:44.456278 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.456251    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.456421 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.456283    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.456421 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.456298    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.456421 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.456312    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df235d02-6ff1-4fe3-8066-83027239875a-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.456421 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.456327    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.457702 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.457676    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-istio-token" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:44.457835 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.457818    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-envoy" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.457941 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.457921    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df235d02-6ff1-4fe3-8066-83027239875a-istio-podinfo" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:59:44.457984 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.457937    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-kube-api-access-sqxfz" pod "df235d02-6ff1-4fe3-8066-83027239875a" (UID: "df235d02-6ff1-4fe3-8066-83027239875a"). InnerVolumeSpecName "kube-api-access-sqxfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:44.458904 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.458883    5817 scope.go:122] "RemoveContainer" containerID="216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0"
Oct 30 21:59:44.459217 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:44.459198    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0\": container with ID starting with 216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0 not found: ID does not exist" containerID="216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0"
Oct 30 21:59:44.459283 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.459235    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0"} err="failed to get container status \"216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0\": rpc error: code = NotFound desc = could not find container \"216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0\": container with ID starting with 216c98295e6c58ac6688c37e41b6149cfbe36016b60e403b221a68c8aec287e0 not found: ID does not exist"
Oct 30 21:59:44.557398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.557335    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.557398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.557394    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df235d02-6ff1-4fe3-8066-83027239875a-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.557398 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.557405    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqxfz\" (UniqueName: \"kubernetes.io/projected/df235d02-6ff1-4fe3-8066-83027239875a-kube-api-access-sqxfz\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.557753 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.557414    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df235d02-6ff1-4fe3-8066-83027239875a-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.735019 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.734994    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:59:44.861508 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.861473    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-envoy\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.861684 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.861571    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-kube-api-access-qxzhl\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-kube-api-access-qxzhl\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.861684 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.861601    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-token\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-token\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.861684 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.861666    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-certs\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.861867 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.861763    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-podinfo\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.862140 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.862112    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-certs" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.862358    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-data\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.862465    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-socket\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.862499    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1160449-671a-4ba2-ae4d-aaa6a3692714-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1160449-671a-4ba2-ae4d-aaa6a3692714-istiod-ca-cert\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.862600    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-data" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.862645    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-credential-socket\") pod \"b1160449-671a-4ba2-ae4d-aaa6a3692714\" (UID: \"b1160449-671a-4ba2-ae4d-aaa6a3692714\") "
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.862777    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-socket" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.863093    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1160449-671a-4ba2-ae4d-aaa6a3692714-istiod-ca-cert" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.863300    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.863316    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-credential-socket" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.863324    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1160449-671a-4ba2-ae4d-aaa6a3692714-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.863340    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.863394 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.863374    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.865777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.864864    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-podinfo" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:59:44.865777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.865062    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-envoy" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:44.865777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.865274    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-token" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:44.865777 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.865338    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-kube-api-access-qxzhl" pod "b1160449-671a-4ba2-ae4d-aaa6a3692714" (UID: "b1160449-671a-4ba2-ae4d-aaa6a3692714"). InnerVolumeSpecName "kube-api-access-qxzhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:44.965532 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.965486    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"]
Oct 30 21:59:44.967306 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.967278    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.967495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.967438    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.967495 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.967484    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxzhl\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-kube-api-access-qxzhl\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.967616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.967500    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.967616 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.967513    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1160449-671a-4ba2-ae4d-aaa6a3692714-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:44.971402 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:44.971375    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-l98th"]
Oct 30 21:59:45.456413 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.456382    5817 generic.go:356] "Generic (PLEG): container finished" podID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerID="e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2" exitCode=0
Oct 30 21:59:45.456804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.456435    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"
Oct 30 21:59:45.456804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.456434    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" event={"ID":"b1160449-671a-4ba2-ae4d-aaa6a3692714","Type":"ContainerDied","Data":"e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2"}
Oct 30 21:59:45.456804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.456551    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5" event={"ID":"b1160449-671a-4ba2-ae4d-aaa6a3692714","Type":"ContainerDied","Data":"8df58d8e70c5b1fa5667cb462d532da9f191a7eba67785844e36bb9997d57684"}
Oct 30 21:59:45.456804 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.456571    5817 scope.go:122] "RemoveContainer" containerID="e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2"
Oct 30 21:59:45.467410 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.467393    5817 scope.go:122] "RemoveContainer" containerID="e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2"
Oct 30 21:59:45.467704 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:45.467684    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2\": container with ID starting with e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2 not found: ID does not exist" containerID="e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2"
Oct 30 21:59:45.467773 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.467717    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2"} err="failed to get container status \"e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2\": rpc error: code = NotFound desc = could not find container \"e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2\": container with ID starting with e8ad034f3cc8ac0934939440e3d13667b446c8ae3db964dd4c25ac1e024d68a2 not found: ID does not exist"
Oct 30 21:59:45.620250 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.620205    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"]
Oct 30 21:59:45.623583 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:45.623557    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-ztjj5"]
Oct 30 21:59:46.338109 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:46.338075    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 21:59:46.844038 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:46.844006    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" path="/var/lib/kubelet/pods/b1160449-671a-4ba2-ae4d-aaa6a3692714/volumes"
Oct 30 21:59:46.844518 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:46.844503    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df235d02-6ff1-4fe3-8066-83027239875a" path="/var/lib/kubelet/pods/df235d02-6ff1-4fe3-8066-83027239875a/volumes"
Oct 30 21:59:49.200383 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:49.200334    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"]
Oct 30 21:59:49.200779 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:49.200560    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy" containerID="cri-o://294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e" gracePeriod=30
Oct 30 21:59:54.469763 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.469695    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:59:54.508498 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508462    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-token\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-token\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.508633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508513    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-credential-socket\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.508633 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508559    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-data\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.508719 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508707    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-certs\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.508830 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508813    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-envoy\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.508881 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508851    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-podinfo\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.508933 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508865    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-credential-socket" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:54.508933 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508877    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-data" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:54.508933 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508906    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istiod-ca-cert\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.509066 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508935    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-kube-api-access-b7p8p\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-kube-api-access-b7p8p\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.509066 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.508985    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-certs" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:54.509066 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509040    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-socket\") pod \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\" (UID: \"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc\") "
Oct 30 21:59:54.509292 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509265    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istiod-ca-cert" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:54.509401 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509377    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-socket" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:54.509889 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509863    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.509986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509895    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.509986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509912    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.509986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509926    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.509986 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.509941    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.511106 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.511084    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-envoy" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:54.511194 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.511109    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-kube-api-access-b7p8p" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "kube-api-access-b7p8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:54.511307 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.511285    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-token" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:54.511469 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.511441    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-podinfo" pod "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" (UID: "3eba6ac8-c7e3-435f-a1d6-e6877cb015bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:59:54.536192 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.536166    5817 generic.go:356] "Generic (PLEG): container finished" podID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerID="294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e" exitCode=0
Oct 30 21:59:54.536315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.536218    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"
Oct 30 21:59:54.536315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.536251    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" event={"ID":"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc","Type":"ContainerDied","Data":"294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e"}
Oct 30 21:59:54.536315 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.536311    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4" event={"ID":"3eba6ac8-c7e3-435f-a1d6-e6877cb015bc","Type":"ContainerDied","Data":"780b055afd6a9766bf268d6ee5deaa5784179c4adb7d8a5e74834abd2681f9e5"}
Oct 30 21:59:54.536859 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.536336    5817 scope.go:122] "RemoveContainer" containerID="294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e"
Oct 30 21:59:54.546840 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.546818    5817 scope.go:122] "RemoveContainer" containerID="183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5"
Oct 30 21:59:54.556641 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.556611    5817 scope.go:122] "RemoveContainer" containerID="294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e"
Oct 30 21:59:54.556932 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:54.556913    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e\": container with ID starting with 294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e not found: ID does not exist" containerID="294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e"
Oct 30 21:59:54.557008 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.556940    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e"} err="failed to get container status \"294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e\": rpc error: code = NotFound desc = could not find container \"294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e\": container with ID starting with 294e543499b1873ff315744f93640c440394625b8c4d6dd371fc59a011fcf14e not found: ID does not exist"
Oct 30 21:59:54.557008 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.556967    5817 scope.go:122] "RemoveContainer" containerID="183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5"
Oct 30 21:59:54.557240 ip-10-0-142-25 kubenswrapper[5817]: E1030 21:59:54.557223    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5\": container with ID starting with 183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5 not found: ID does not exist" containerID="183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5"
Oct 30 21:59:54.557290 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.557246    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5"} err="failed to get container status \"183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5\": rpc error: code = NotFound desc = could not find container \"183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5\": container with ID starting with 183a78c607ccd4a4525882df899f0534fa9dfb72d6d0968d67d7f83371dbdac5 not found: ID does not exist"
Oct 30 21:59:54.611072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.611041    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.611072 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.611068    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.611280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.611080    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7p8p\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-kube-api-access-b7p8p\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.611280 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.611090    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:54.783743 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.783645    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"]
Oct 30 21:59:54.790627 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.790594    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-737a64-openshift-default-69bf645c8f-crdm4"]
Oct 30 21:59:54.845231 ip-10-0-142-25 kubenswrapper[5817]: I1030 21:59:54.845191    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" path="/var/lib/kubelet/pods/3eba6ac8-c7e3-435f-a1d6-e6877cb015bc/volumes"
Oct 30 22:00:00.220091 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220057    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"]
Oct 30 22:00:00.220796 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220779    5817 cpu_manager.go:414] "containerMap: removing container" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy"
Oct 30 22:00:00.220848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220798    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy"
Oct 30 22:00:00.220848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220811    5817 cpu_manager.go:414] "containerMap: removing container" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy"
Oct 30 22:00:00.220848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220816    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy"
Oct 30 22:00:00.220848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220826    5817 cpu_manager.go:414] "containerMap: removing container" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main"
Oct 30 22:00:00.220848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220831    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main"
Oct 30 22:00:00.220848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220843    5817 cpu_manager.go:414] "containerMap: removing container" podUID="b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" containerName="maas-api"
Oct 30 22:00:00.220848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220848    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" containerName="maas-api"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220857    5817 cpu_manager.go:414] "containerMap: removing container" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220863    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220877    5817 cpu_manager.go:414] "containerMap: removing container" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220881    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220889    5817 cpu_manager.go:414] "containerMap: removing container" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220893    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220899    5817 cpu_manager.go:414] "containerMap: removing container" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220903    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220912    5817 cpu_manager.go:414] "containerMap: removing container" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220916    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220922    5817 cpu_manager.go:414] "containerMap: removing container" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220926    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220933    5817 cpu_manager.go:414] "containerMap: removing container" podUID="a52eda1a-e83b-4967-9410-15b96a743607" containerName="maas-api"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220937    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a52eda1a-e83b-4967-9410-15b96a743607" containerName="maas-api"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220950    5817 cpu_manager.go:414] "containerMap: removing container" podUID="dfb8c540-571e-4f74-99c0-a6dc852fe900" containerName="payload-pre-processing"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220955    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfb8c540-571e-4f74-99c0-a6dc852fe900" containerName="payload-pre-processing"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220962    5817 cpu_manager.go:414] "containerMap: removing container" podUID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" containerName="payload-processing"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220966    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" containerName="payload-processing"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220974    5817 cpu_manager.go:414] "containerMap: removing container" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.220978    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221004    5817 cpu_manager.go:414] "containerMap: removing container" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy"
Oct 30 22:00:00.221072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221008    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221106    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221116    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f17ad69-4769-4a86-84da-5c3f9f2df240" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221122    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221128    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dcbc03b-703e-4e1c-a76c-339a0bc312a3" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221134    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221141    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="a52eda1a-e83b-4967-9410-15b96a743607" containerName="maas-api"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221148    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfb8c540-571e-4f74-99c0-a6dc852fe900" containerName="payload-pre-processing"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221154    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="f49b2817-d5d6-438f-819a-58a8fbc69e3d" containerName="payload-processing"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221160    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1160449-671a-4ba2-ae4d-aaa6a3692714" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221168    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="df235d02-6ff1-4fe3-8066-83027239875a" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221173    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="544e7dca-6988-491c-907b-e903dafe5c65" containerName="istio-proxy"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221179    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="865b13f8-87cb-496f-b90f-080ed2aaff6e" containerName="main"
Oct 30 22:00:00.222021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.221184    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7cad3a9-6764-4474-9e26-d8e5b9abc0e9" containerName="maas-api"
Oct 30 22:00:00.227479 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.227452    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.246591 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.246563    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"]
Oct 30 22:00:00.247495 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.247463    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="19cac82a-c4ed-4d20-8ed2-07d1bd6e3c5d" containerName="istio-proxy"
Oct 30 22:00:00.247495 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.247496    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="3eba6ac8-c7e3-435f-a1d6-e6877cb015bc" containerName="istio-proxy"
Oct 30 22:00:00.251118 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.251096    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.275749 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.275542    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"]
Oct 30 22:00:00.280589 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.280565    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.304720 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.304688    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"]
Oct 30 22:00:00.309847 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.309566    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.309847 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.309744    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md52j\" (UniqueName: \"kubernetes.io/projected/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-kube-api-access-md52j\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.310111 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.310092    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"]
Oct 30 22:00:00.310241 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.310222    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.316794 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.315313    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.324503 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.324476    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"]
Oct 30 22:00:00.346416 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.345201    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"]
Oct 30 22:00:00.351300 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.351263    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c6npn\""
Oct 30 22:00:00.352803 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.352776    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"]
Oct 30 22:00:00.398879 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.398847    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"]
Oct 30 22:00:00.402725 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.402690    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"]
Oct 30 22:00:00.411439 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.411406    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md52j\" (UniqueName: \"kubernetes.io/projected/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-kube-api-access-md52j\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.411607 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.411463    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldpfn\" (UniqueName: \"kubernetes.io/projected/245c8d07-a644-42c5-9414-b2d1611f20e5-kube-api-access-ldpfn\") pod \"maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.411607 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.411592    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bkhh\" (UniqueName: \"kubernetes.io/projected/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-kube-api-access-4bkhh\") pod \"maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.411779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.411657    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.411894 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.411875    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cht7\" (UniqueName: \"kubernetes.io/projected/3955d3d7-0b83-487f-846f-498f29975729-kube-api-access-4cht7\") pod \"maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.411962 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.411904    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.412022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.411972    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80f2967-f96a-46b6-824c-f7bb349a17a7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.412080 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.412041    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3955d3d7-0b83-487f-846f-498f29975729-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.412134 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.412081    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/245c8d07-a644-42c5-9414-b2d1611f20e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.412199 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.412177    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb7kc\" (UniqueName: \"kubernetes.io/projected/e80f2967-f96a-46b6-824c-f7bb349a17a7-kube-api-access-cb7kc\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.412657 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.412636    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.476829 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.476764    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md52j\" (UniqueName: \"kubernetes.io/projected/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-kube-api-access-md52j\") pod \"maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.513534 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513500    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3955d3d7-0b83-487f-846f-498f29975729-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.513718 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513561    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/245c8d07-a644-42c5-9414-b2d1611f20e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.513718 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513608    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb7kc\" (UniqueName: \"kubernetes.io/projected/e80f2967-f96a-46b6-824c-f7bb349a17a7-kube-api-access-cb7kc\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.513718 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513647    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldpfn\" (UniqueName: \"kubernetes.io/projected/245c8d07-a644-42c5-9414-b2d1611f20e5-kube-api-access-ldpfn\") pod \"maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.513718 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513697    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bkhh\" (UniqueName: \"kubernetes.io/projected/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-kube-api-access-4bkhh\") pod \"maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.513968 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513751    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.513968 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513888    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cht7\" (UniqueName: \"kubernetes.io/projected/3955d3d7-0b83-487f-846f-498f29975729-kube-api-access-4cht7\") pod \"maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.513968 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.513950    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80f2967-f96a-46b6-824c-f7bb349a17a7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.514331 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.514306    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3955d3d7-0b83-487f-846f-498f29975729-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.514443 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.514338    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/245c8d07-a644-42c5-9414-b2d1611f20e5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.514514 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.514494    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.514604 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.514579    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80f2967-f96a-46b6-824c-f7bb349a17a7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.542914 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.542890    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:00.583163 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.583136    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bkhh\" (UniqueName: \"kubernetes.io/projected/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-kube-api-access-4bkhh\") pod \"maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.583319 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.583143    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cht7\" (UniqueName: \"kubernetes.io/projected/3955d3d7-0b83-487f-846f-498f29975729-kube-api-access-4cht7\") pod \"maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.586828 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.586804    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb7kc\" (UniqueName: \"kubernetes.io/projected/e80f2967-f96a-46b6-824c-f7bb349a17a7-kube-api-access-cb7kc\") pod \"maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.588252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.588232    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldpfn\" (UniqueName: \"kubernetes.io/projected/245c8d07-a644-42c5-9414-b2d1611f20e5-kube-api-access-ldpfn\") pod \"maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.600191 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.600167    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:00.635804 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.632696    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:00.646116 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.645931    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:00.869222 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.869185    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:00.936922 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:00.936645    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"]
Oct 30 22:00:00.940326 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:00.940163    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fcfc53c_dd7d_45fb_879d_2da556c1b4c1.slice/crio-8f55e7e589aeaf8193789c2120e4980aed029379a9bc1fbcd935561586c178e7 WatchSource:0}: Error finding container 8f55e7e589aeaf8193789c2120e4980aed029379a9bc1fbcd935561586c178e7: Status 404 returned error can't find the container with id 8f55e7e589aeaf8193789c2120e4980aed029379a9bc1fbcd935561586c178e7
Oct 30 22:00:01.015865 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.015819    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"]
Oct 30 22:00:01.019116 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:01.019088    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80f2967_f96a_46b6_824c_f7bb349a17a7.slice/crio-097b6849242924e7f464819aef9f53f0e101ad05d233934ff6ee48a8d1466654 WatchSource:0}: Error finding container 097b6849242924e7f464819aef9f53f0e101ad05d233934ff6ee48a8d1466654: Status 404 returned error can't find the container with id 097b6849242924e7f464819aef9f53f0e101ad05d233934ff6ee48a8d1466654
Oct 30 22:00:01.077067 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.077040    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"]
Oct 30 22:00:01.078628 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:01.078600    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3955d3d7_0b83_487f_846f_498f29975729.slice/crio-fb3aad24f3b161d7c6af308e8db515d403ce91fc30b1d83d9e376a30944d6062 WatchSource:0}: Error finding container fb3aad24f3b161d7c6af308e8db515d403ce91fc30b1d83d9e376a30944d6062: Status 404 returned error can't find the container with id fb3aad24f3b161d7c6af308e8db515d403ce91fc30b1d83d9e376a30944d6062
Oct 30 22:00:01.100260 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.100233    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"]
Oct 30 22:00:01.101153 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:01.101130    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f0ee84_2b7d_4258_96ba_d9204cdc5089.slice/crio-0ebf1c9ca0a11bfea6edc6cec4bb20dadf6f5a61edc2cce20fd434404d7abade WatchSource:0}: Error finding container 0ebf1c9ca0a11bfea6edc6cec4bb20dadf6f5a61edc2cce20fd434404d7abade: Status 404 returned error can't find the container with id 0ebf1c9ca0a11bfea6edc6cec4bb20dadf6f5a61edc2cce20fd434404d7abade
Oct 30 22:00:01.256329 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.256182    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"]
Oct 30 22:00:01.259885 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:01.259854    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245c8d07_a644_42c5_9414_b2d1611f20e5.slice/crio-d573f4b950f8aa135d1a7a337e67be6eecaf5d6e3053d3ef0e7b6a58ee76b5b5 WatchSource:0}: Error finding container d573f4b950f8aa135d1a7a337e67be6eecaf5d6e3053d3ef0e7b6a58ee76b5b5: Status 404 returned error can't find the container with id d573f4b950f8aa135d1a7a337e67be6eecaf5d6e3053d3ef0e7b6a58ee76b5b5
Oct 30 22:00:01.593861 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.593817    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" event={"ID":"3955d3d7-0b83-487f-846f-498f29975729","Type":"ContainerStarted","Data":"fb3aad24f3b161d7c6af308e8db515d403ce91fc30b1d83d9e376a30944d6062"}
Oct 30 22:00:01.595676 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.595646    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" event={"ID":"e80f2967-f96a-46b6-824c-f7bb349a17a7","Type":"ContainerStarted","Data":"43f68de656dce58596c16a2c79675cf5dd8d3b93963555fac1a921bfaaf90473"}
Oct 30 22:00:01.595804 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.595705    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" event={"ID":"e80f2967-f96a-46b6-824c-f7bb349a17a7","Type":"ContainerStarted","Data":"097b6849242924e7f464819aef9f53f0e101ad05d233934ff6ee48a8d1466654"}
Oct 30 22:00:01.597024 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.597001    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" event={"ID":"245c8d07-a644-42c5-9414-b2d1611f20e5","Type":"ContainerStarted","Data":"d573f4b950f8aa135d1a7a337e67be6eecaf5d6e3053d3ef0e7b6a58ee76b5b5"}
Oct 30 22:00:01.598212 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.598193    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" event={"ID":"e3f0ee84-2b7d-4258-96ba-d9204cdc5089","Type":"ContainerStarted","Data":"0ebf1c9ca0a11bfea6edc6cec4bb20dadf6f5a61edc2cce20fd434404d7abade"}
Oct 30 22:00:01.599715 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.599690    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerStarted","Data":"f2d2b90370562512f5496e7ec8e53e4eee4bd94698e958a8e8201a4cd9a7eccd"}
Oct 30 22:00:01.599811 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.599739    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerStarted","Data":"8f55e7e589aeaf8193789c2120e4980aed029379a9bc1fbcd935561586c178e7"}
Oct 30 22:00:01.710728 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.710671    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" podStartSLOduration=1.710648968 podStartE2EDuration="1.710648968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:01.706628807 +0000 UTC m=+2736.994351367" watchObservedRunningTime="2026-10-30 22:00:01.710648968 +0000 UTC m=+2736.998371515"
Oct 30 22:00:01.822014 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:01.821899    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" podStartSLOduration=1.821886125 podStartE2EDuration="1.821886125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:01.821483752 +0000 UTC m=+2737.109206297" watchObservedRunningTime="2026-10-30 22:00:01.821886125 +0000 UTC m=+2737.109608672"
Oct 30 22:00:02.608858 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.608821    5817 generic.go:356] "Generic (PLEG): container finished" podID="3955d3d7-0b83-487f-846f-498f29975729" containerID="e9ad3094920ddcd6fcac267ca73dbca5199a2f709ebff41f12de0c1971acb083" exitCode=60
Oct 30 22:00:02.609302 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.608916    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" event={"ID":"3955d3d7-0b83-487f-846f-498f29975729","Type":"ContainerDied","Data":"e9ad3094920ddcd6fcac267ca73dbca5199a2f709ebff41f12de0c1971acb083"}
Oct 30 22:00:02.609302 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.609227    5817 scope.go:122] "RemoveContainer" containerID="e9ad3094920ddcd6fcac267ca73dbca5199a2f709ebff41f12de0c1971acb083"
Oct 30 22:00:02.610587 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.610565    5817 generic.go:356] "Generic (PLEG): container finished" podID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerID="43f68de656dce58596c16a2c79675cf5dd8d3b93963555fac1a921bfaaf90473" exitCode=60
Oct 30 22:00:02.610722 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.610652    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" event={"ID":"e80f2967-f96a-46b6-824c-f7bb349a17a7","Type":"ContainerDied","Data":"43f68de656dce58596c16a2c79675cf5dd8d3b93963555fac1a921bfaaf90473"}
Oct 30 22:00:02.610912 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.610883    5817 scope.go:122] "RemoveContainer" containerID="43f68de656dce58596c16a2c79675cf5dd8d3b93963555fac1a921bfaaf90473"
Oct 30 22:00:02.612383 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.612340    5817 generic.go:356] "Generic (PLEG): container finished" podID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerID="ee2574ec730367df5161ff52727ef893e5648ff6ef3c522d6968854ee92ca116" exitCode=60
Oct 30 22:00:02.612484 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.612450    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" event={"ID":"245c8d07-a644-42c5-9414-b2d1611f20e5","Type":"ContainerDied","Data":"ee2574ec730367df5161ff52727ef893e5648ff6ef3c522d6968854ee92ca116"}
Oct 30 22:00:02.612715 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.612690    5817 scope.go:122] "RemoveContainer" containerID="ee2574ec730367df5161ff52727ef893e5648ff6ef3c522d6968854ee92ca116"
Oct 30 22:00:02.614164 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.614144    5817 generic.go:356] "Generic (PLEG): container finished" podID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerID="fd6362c6b9c60364beeecbc3a90d02a5316b9891b9833ccf905715ccee15feeb" exitCode=60
Oct 30 22:00:02.614422 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.614276    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" event={"ID":"e3f0ee84-2b7d-4258-96ba-d9204cdc5089","Type":"ContainerDied","Data":"fd6362c6b9c60364beeecbc3a90d02a5316b9891b9833ccf905715ccee15feeb"}
Oct 30 22:00:02.614476 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.614421    5817 scope.go:122] "RemoveContainer" containerID="fd6362c6b9c60364beeecbc3a90d02a5316b9891b9833ccf905715ccee15feeb"
Oct 30 22:00:02.616008 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.615990    5817 generic.go:356] "Generic (PLEG): container finished" podID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerID="f2d2b90370562512f5496e7ec8e53e4eee4bd94698e958a8e8201a4cd9a7eccd" exitCode=60
Oct 30 22:00:02.616102 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.616036    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerDied","Data":"f2d2b90370562512f5496e7ec8e53e4eee4bd94698e958a8e8201a4cd9a7eccd"}
Oct 30 22:00:02.616289 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:02.616273    5817 scope.go:122] "RemoveContainer" containerID="f2d2b90370562512f5496e7ec8e53e4eee4bd94698e958a8e8201a4cd9a7eccd"
Oct 30 22:00:03.625655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:03.625615    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerStarted","Data":"171f0898644ec5715687b184a5700ab92ceb9ed9e063cecf89b2857a0cfba23f"}
Oct 30 22:00:03.627805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:03.627776    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" event={"ID":"3955d3d7-0b83-487f-846f-498f29975729","Type":"ContainerStarted","Data":"8f579a927c2c99ed0446484dd131f5089c3492470f6bbc7de652f6a994cddce1"}
Oct 30 22:00:03.629615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:03.629581    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" event={"ID":"e80f2967-f96a-46b6-824c-f7bb349a17a7","Type":"ContainerStarted","Data":"d33b1ca8b6ccb9d83ab9184629fbb568012c69a87759996852385f7d17a5d894"}
Oct 30 22:00:03.631502 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:03.631475    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" event={"ID":"245c8d07-a644-42c5-9414-b2d1611f20e5","Type":"ContainerStarted","Data":"286aff5426a768ca2abeef28781508dbc09acbc48e041ce04b60a4bbcbc8c4d4"}
Oct 30 22:00:03.633481 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:03.633454    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" event={"ID":"e3f0ee84-2b7d-4258-96ba-d9204cdc5089","Type":"ContainerStarted","Data":"75ea90cbd32cf35f57ae8f18e07043256f47056b1455dacfa6f6ed8f8fe14c23"}
Oct 30 22:00:03.869015 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:03.868965    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" podStartSLOduration=3.868950489 podStartE2EDuration="3.868950489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:03.865693667 +0000 UTC m=+2739.153416213" watchObservedRunningTime="2026-10-30 22:00:03.868950489 +0000 UTC m=+2739.156673034"
Oct 30 22:00:03.987424 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:03.987336    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" podStartSLOduration=3.9873219 podStartE2EDuration="3.9873219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:03.983932932 +0000 UTC m=+2739.271655478" watchObservedRunningTime="2026-10-30 22:00:03.9873219 +0000 UTC m=+2739.275044440"
Oct 30 22:00:04.097916 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.097858    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" podStartSLOduration=4.097843402 podStartE2EDuration="4.097843402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:04.096842789 +0000 UTC m=+2739.384565336" watchObservedRunningTime="2026-10-30 22:00:04.097843402 +0000 UTC m=+2739.385565947"
Oct 30 22:00:04.624326 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.624280    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"]
Oct 30 22:00:04.642714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.642678    5817 generic.go:356] "Generic (PLEG): container finished" podID="3955d3d7-0b83-487f-846f-498f29975729" containerID="8f579a927c2c99ed0446484dd131f5089c3492470f6bbc7de652f6a994cddce1" exitCode=60
Oct 30 22:00:04.643167 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.642762    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" event={"ID":"3955d3d7-0b83-487f-846f-498f29975729","Type":"ContainerDied","Data":"8f579a927c2c99ed0446484dd131f5089c3492470f6bbc7de652f6a994cddce1"}
Oct 30 22:00:04.643167 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.642835    5817 scope.go:122] "RemoveContainer" containerID="e9ad3094920ddcd6fcac267ca73dbca5199a2f709ebff41f12de0c1971acb083"
Oct 30 22:00:04.643167 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.643072    5817 scope.go:122] "RemoveContainer" containerID="8f579a927c2c99ed0446484dd131f5089c3492470f6bbc7de652f6a994cddce1"
Oct 30 22:00:04.643578 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:04.643524    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7_odh-ai-gateway-infra(3955d3d7-0b83-487f-846f-498f29975729)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" podUID="3955d3d7-0b83-487f-846f-498f29975729"
Oct 30 22:00:04.645554 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.645534    5817 generic.go:356] "Generic (PLEG): container finished" podID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerID="d33b1ca8b6ccb9d83ab9184629fbb568012c69a87759996852385f7d17a5d894" exitCode=60
Oct 30 22:00:04.645681 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.645602    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" event={"ID":"e80f2967-f96a-46b6-824c-f7bb349a17a7","Type":"ContainerDied","Data":"d33b1ca8b6ccb9d83ab9184629fbb568012c69a87759996852385f7d17a5d894"}
Oct 30 22:00:04.645945 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.645927    5817 scope.go:122] "RemoveContainer" containerID="d33b1ca8b6ccb9d83ab9184629fbb568012c69a87759996852385f7d17a5d894"
Oct 30 22:00:04.646294 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:04.646270    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk_odh-ai-gateway-infra(e80f2967-f96a-46b6-824c-f7bb349a17a7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7"
Oct 30 22:00:04.647684 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.647665    5817 generic.go:356] "Generic (PLEG): container finished" podID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerID="286aff5426a768ca2abeef28781508dbc09acbc48e041ce04b60a4bbcbc8c4d4" exitCode=60
Oct 30 22:00:04.647785 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.647732    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" event={"ID":"245c8d07-a644-42c5-9414-b2d1611f20e5","Type":"ContainerDied","Data":"286aff5426a768ca2abeef28781508dbc09acbc48e041ce04b60a4bbcbc8c4d4"}
Oct 30 22:00:04.648057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.648041    5817 scope.go:122] "RemoveContainer" containerID="286aff5426a768ca2abeef28781508dbc09acbc48e041ce04b60a4bbcbc8c4d4"
Oct 30 22:00:04.648373 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:04.648338    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6_odh-ai-gateway-infra(245c8d07-a644-42c5-9414-b2d1611f20e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5"
Oct 30 22:00:04.649720 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.649700    5817 generic.go:356] "Generic (PLEG): container finished" podID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerID="75ea90cbd32cf35f57ae8f18e07043256f47056b1455dacfa6f6ed8f8fe14c23" exitCode=60
Oct 30 22:00:04.649800 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.649773    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" event={"ID":"e3f0ee84-2b7d-4258-96ba-d9204cdc5089","Type":"ContainerDied","Data":"75ea90cbd32cf35f57ae8f18e07043256f47056b1455dacfa6f6ed8f8fe14c23"}
Oct 30 22:00:04.650035 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.650019    5817 scope.go:122] "RemoveContainer" containerID="75ea90cbd32cf35f57ae8f18e07043256f47056b1455dacfa6f6ed8f8fe14c23"
Oct 30 22:00:04.650333 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:04.650312    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9_odh-ai-gateway-infra(e3f0ee84-2b7d-4258-96ba-d9204cdc5089)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089"
Oct 30 22:00:04.652285 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.652266    5817 generic.go:356] "Generic (PLEG): container finished" podID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerID="171f0898644ec5715687b184a5700ab92ceb9ed9e063cecf89b2857a0cfba23f" exitCode=60
Oct 30 22:00:04.652407 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.652319    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerDied","Data":"171f0898644ec5715687b184a5700ab92ceb9ed9e063cecf89b2857a0cfba23f"}
Oct 30 22:00:04.652627 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.652607    5817 scope.go:122] "RemoveContainer" containerID="171f0898644ec5715687b184a5700ab92ceb9ed9e063cecf89b2857a0cfba23f"
Oct 30 22:00:04.652928 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:04.652905    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8_odh-ai-gateway-infra(5fcfc53c-dd7d-45fb-879d-2da556c1b4c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1"
Oct 30 22:00:04.657274 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.657251    5817 scope.go:122] "RemoveContainer" containerID="43f68de656dce58596c16a2c79675cf5dd8d3b93963555fac1a921bfaaf90473"
Oct 30 22:00:04.672413 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.672390    5817 scope.go:122] "RemoveContainer" containerID="ee2574ec730367df5161ff52727ef893e5648ff6ef3c522d6968854ee92ca116"
Oct 30 22:00:04.685625 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.685603    5817 scope.go:122] "RemoveContainer" containerID="fd6362c6b9c60364beeecbc3a90d02a5316b9891b9833ccf905715ccee15feeb"
Oct 30 22:00:04.698713 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:04.698693    5817 scope.go:122] "RemoveContainer" containerID="f2d2b90370562512f5496e7ec8e53e4eee4bd94698e958a8e8201a4cd9a7eccd"
Oct 30 22:00:05.576667 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.576629    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"]
Oct 30 22:00:05.582256 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.582233    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.644034 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.644005    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"]
Oct 30 22:00:05.663927 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.663899    5817 scope.go:122] "RemoveContainer" containerID="171f0898644ec5715687b184a5700ab92ceb9ed9e063cecf89b2857a0cfba23f"
Oct 30 22:00:05.664322 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:05.664296    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8_odh-ai-gateway-infra(5fcfc53c-dd7d-45fb-879d-2da556c1b4c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1"
Oct 30 22:00:05.665896 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.665848    5817 scope.go:122] "RemoveContainer" containerID="8f579a927c2c99ed0446484dd131f5089c3492470f6bbc7de652f6a994cddce1"
Oct 30 22:00:05.666235 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:05.666213    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7_odh-ai-gateway-infra(3955d3d7-0b83-487f-846f-498f29975729)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" podUID="3955d3d7-0b83-487f-846f-498f29975729"
Oct 30 22:00:05.667637 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.667615    5817 scope.go:122] "RemoveContainer" containerID="d33b1ca8b6ccb9d83ab9184629fbb568012c69a87759996852385f7d17a5d894"
Oct 30 22:00:05.667954 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:05.667927    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk_odh-ai-gateway-infra(e80f2967-f96a-46b6-824c-f7bb349a17a7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7"
Oct 30 22:00:05.669335 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.669316    5817 scope.go:122] "RemoveContainer" containerID="286aff5426a768ca2abeef28781508dbc09acbc48e041ce04b60a4bbcbc8c4d4"
Oct 30 22:00:05.669659 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:05.669634    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6_odh-ai-gateway-infra(245c8d07-a644-42c5-9414-b2d1611f20e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5"
Oct 30 22:00:05.671071 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.671047    5817 scope.go:122] "RemoveContainer" containerID="75ea90cbd32cf35f57ae8f18e07043256f47056b1455dacfa6f6ed8f8fe14c23"
Oct 30 22:00:05.671399 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:05.671336    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9_odh-ai-gateway-infra(e3f0ee84-2b7d-4258-96ba-d9204cdc5089)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089"
Oct 30 22:00:05.705155 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705120    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705283 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705177    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsmbv\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-kube-api-access-gsmbv\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705283 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705256    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705436 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705408    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705508 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705462    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705567 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705552    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705579    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705600    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.705745 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.705725    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.806866 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.806826    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807041 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.806932    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807041 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807004    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807055    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807102    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807161    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807342 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807261    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807646 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807619    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807717 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807641    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807788 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807738    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsmbv\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-kube-api-access-gsmbv\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807856 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807837    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.807935 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.807911    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.808461 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.808227    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.808917 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.808895    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.810014 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.809988    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.810286 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.810267    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.872561 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.872476    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsmbv\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-kube-api-access-gsmbv\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.872985 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.872954    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:05.896580 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:05.896555    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:06.355949 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.355905    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"]
Oct 30 22:00:06.356565 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:06.356460    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-32acc6dafaab1a0dd3b86874374fe44d81ae9383fbb7d4927cb1f9382924529b WatchSource:0}: Error finding container 32acc6dafaab1a0dd3b86874374fe44d81ae9383fbb7d4927cb1f9382924529b: Status 404 returned error can't find the container with id 32acc6dafaab1a0dd3b86874374fe44d81ae9383fbb7d4927cb1f9382924529b
Oct 30 22:00:06.361799 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.361740    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:06.364384 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.361940    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:06.364384 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.362081    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:06.680334 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.680243    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" event={"ID":"9680477d-1d4e-45b5-b96b-f20a0044fd3e","Type":"ContainerStarted","Data":"0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad"}
Oct 30 22:00:06.680334 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.680317    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" event={"ID":"9680477d-1d4e-45b5-b96b-f20a0044fd3e","Type":"ContainerStarted","Data":"32acc6dafaab1a0dd3b86874374fe44d81ae9383fbb7d4927cb1f9382924529b"}
Oct 30 22:00:06.812107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.812055    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" podStartSLOduration=1.812037924 podStartE2EDuration="1.812037924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:06.811089104 +0000 UTC m=+2742.098811654" watchObservedRunningTime="2026-10-30 22:00:06.812037924 +0000 UTC m=+2742.099760470"
Oct 30 22:00:06.897471 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:06.897431    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:07.914721 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:07.914675    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 22:00:07.915146 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:07.914748    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 22:00:08.516555 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:08.516519    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"]
Oct 30 22:00:08.516762 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:08.516732    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy" containerID="cri-o://1d21dc51ed948166c592ae735d5e871ffa5835601abfa28cc8be361920c40bad" gracePeriod=30
Oct 30 22:00:08.909429 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:08.902513    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 22:00:08.909429 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:08.902575    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 22:00:09.625195 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:09.625155    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r" podUID="f7fa220f-5ded-45b4-94b5-400893ffdac5" containerName="payload-pre-processing" containerID="cri-o://9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54" gracePeriod=25
Oct 30 22:00:09.920137 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:09.919826    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 22:00:09.920137 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:09.919894    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 22:00:09.973368 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:09.972846    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 22:00:10.079228 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.079143    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7fa220f-5ded-45b4-94b5-400893ffdac5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fa220f-5ded-45b4-94b5-400893ffdac5-plugins-config-volume\") pod \"f7fa220f-5ded-45b4-94b5-400893ffdac5\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") "
Oct 30 22:00:10.079772 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.079652    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fa220f-5ded-45b4-94b5-400893ffdac5-plugins-config-volume" pod "f7fa220f-5ded-45b4-94b5-400893ffdac5" (UID: "f7fa220f-5ded-45b4-94b5-400893ffdac5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:10.080057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.079975    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7fa220f-5ded-45b4-94b5-400893ffdac5-kube-api-access-wknjc\" (UniqueName: \"kubernetes.io/projected/f7fa220f-5ded-45b4-94b5-400893ffdac5-kube-api-access-wknjc\") pod \"f7fa220f-5ded-45b4-94b5-400893ffdac5\" (UID: \"f7fa220f-5ded-45b4-94b5-400893ffdac5\") "
Oct 30 22:00:10.081074 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.081019    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7fa220f-5ded-45b4-94b5-400893ffdac5-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:10.084584 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.084558    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fa220f-5ded-45b4-94b5-400893ffdac5-kube-api-access-wknjc" pod "f7fa220f-5ded-45b4-94b5-400893ffdac5" (UID: "f7fa220f-5ded-45b4-94b5-400893ffdac5"). InnerVolumeSpecName "kube-api-access-wknjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:10.182767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.182678    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wknjc\" (UniqueName: \"kubernetes.io/projected/f7fa220f-5ded-45b4-94b5-400893ffdac5-kube-api-access-wknjc\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:10.738085 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.738050    5817 generic.go:356] "Generic (PLEG): container finished" podID="f7fa220f-5ded-45b4-94b5-400893ffdac5" containerID="9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54" exitCode=0
Oct 30 22:00:10.738598 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.738097    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r" event={"ID":"f7fa220f-5ded-45b4-94b5-400893ffdac5","Type":"ContainerDied","Data":"9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54"}
Oct 30 22:00:10.738598 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.738107    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"
Oct 30 22:00:10.738598 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.738137    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r" event={"ID":"f7fa220f-5ded-45b4-94b5-400893ffdac5","Type":"ContainerDied","Data":"8ac333ed2a016d5c15674ce75217bd2d64b0a2d66dd0ea818ff0c7bf9e129f6c"}
Oct 30 22:00:10.738598 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.738159    5817 scope.go:122] "RemoveContainer" containerID="9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54"
Oct 30 22:00:10.749322 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.749299    5817 scope.go:122] "RemoveContainer" containerID="9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54"
Oct 30 22:00:10.749625 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:10.749606    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54\": container with ID starting with 9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54 not found: ID does not exist" containerID="9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54"
Oct 30 22:00:10.749699 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.749636    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54"} err="failed to get container status \"9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54\": rpc error: code = NotFound desc = could not find container \"9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54\": container with ID starting with 9dc4552101b173a14bbb4e2e0697452b97207a05c80ead3ac5b7e87981648e54 not found: ID does not exist"
Oct 30 22:00:10.905374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.901818    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 22:00:10.905374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.901883    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 22:00:10.937817 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.937727    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"]
Oct 30 22:00:10.945518 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:10.944124    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-40bb78-5499df44c-hll5r"]
Oct 30 22:00:11.905185 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:11.904709    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 22:00:11.905185 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:11.904787    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 22:00:12.847678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:12.847633    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7fa220f-5ded-45b4-94b5-400893ffdac5" path="/var/lib/kubelet/pods/f7fa220f-5ded-45b4-94b5-400893ffdac5/volumes"
Oct 30 22:00:12.931005 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:12.930966    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:12.931587 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:12.931381    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:12.932764 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:12.932736    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:00:13.764525 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.764488    5817 generic.go:356] "Generic (PLEG): container finished" podID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerID="1d21dc51ed948166c592ae735d5e871ffa5835601abfa28cc8be361920c40bad" exitCode=0
Oct 30 22:00:13.764722 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.764564    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" event={"ID":"d4f91b4f-7f3f-492a-aaea-d71076cde8c5","Type":"ContainerDied","Data":"1d21dc51ed948166c592ae735d5e871ffa5835601abfa28cc8be361920c40bad"}
Oct 30 22:00:13.764722 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.764640    5817 scope.go:122] "RemoveContainer" containerID="fa612a3be8b390420799ca6d42c486dd996d1fe882ccfb3b7eaf73ffb7f6c1c7"
Oct 30 22:00:13.814511 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.814487    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 22:00:13.836706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.836668    5817 scope.go:122] "RemoveContainer" containerID="8f579a927c2c99ed0446484dd131f5089c3492470f6bbc7de652f6a994cddce1"
Oct 30 22:00:13.869978 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.869948    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-socket\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870095 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870014    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-certs\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870095 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870059    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istiod-ca-cert\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870218 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870185    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-envoy\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870327 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870203    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-socket" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:13.870327 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870257    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-data\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870327 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870289    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-token\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-token\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870331    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-certs" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:13.870504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870397    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-credential-socket\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870439    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-kube-api-access-24sp8\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-kube-api-access-24sp8\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870467    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-podinfo\") pod \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\" (UID: \"d4f91b4f-7f3f-492a-aaea-d71076cde8c5\") "
Oct 30 22:00:13.870712 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870504    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-data" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:13.870766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870712    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-credential-socket" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:13.872039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.870886    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istiod-ca-cert" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:13.872039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.871516    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.872039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.871539    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.872039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.871554    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.872039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.871570    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.872039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.871585    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.872846 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.872826    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-envoy" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:13.873044 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.873024    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-token" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:13.873106 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.873061    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-kube-api-access-24sp8" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "kube-api-access-24sp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:13.873106 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.873083    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-podinfo" pod "d4f91b4f-7f3f-492a-aaea-d71076cde8c5" (UID: "d4f91b4f-7f3f-492a-aaea-d71076cde8c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:00:13.972511 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.972481    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.972511 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.972509    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.973041 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.972521    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24sp8\" (UniqueName: \"kubernetes.io/projected/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-kube-api-access-24sp8\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:13.973041 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:13.972536    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f91b4f-7f3f-492a-aaea-d71076cde8c5-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:14.773433 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.773392    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9" event={"ID":"d4f91b4f-7f3f-492a-aaea-d71076cde8c5","Type":"ContainerDied","Data":"e2bfde75ee5c13b7917cf6b75bb3aecd18cd920ecc0a86f8262fe37f2984de1c"}
Oct 30 22:00:14.773651 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.773465    5817 scope.go:122] "RemoveContainer" containerID="1d21dc51ed948166c592ae735d5e871ffa5835601abfa28cc8be361920c40bad"
Oct 30 22:00:14.773651 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.773516    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"
Oct 30 22:00:14.776477 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.776440    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" event={"ID":"3955d3d7-0b83-487f-846f-498f29975729","Type":"ContainerStarted","Data":"d9026efde5dc4e3eda96aa094b4404ad82f6a01d6af78335ed2fb0630f6177c4"}
Oct 30 22:00:14.836015 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.835983    5817 scope.go:122] "RemoveContainer" containerID="171f0898644ec5715687b184a5700ab92ceb9ed9e063cecf89b2857a0cfba23f"
Oct 30 22:00:14.836171 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.836148    5817 scope.go:122] "RemoveContainer" containerID="286aff5426a768ca2abeef28781508dbc09acbc48e041ce04b60a4bbcbc8c4d4"
Oct 30 22:00:14.938407 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.938370    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"]
Oct 30 22:00:14.985168 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:14.985130    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-167561-openshift-default-7878bdf957-xt4t9"]
Oct 30 22:00:15.784972 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:15.784935    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerStarted","Data":"3a6001aaf488add578535c39bb63dd1e25b6767200579e0cc02fc3835431e5fa"}
Oct 30 22:00:15.788146 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:15.788118    5817 generic.go:356] "Generic (PLEG): container finished" podID="3955d3d7-0b83-487f-846f-498f29975729" containerID="d9026efde5dc4e3eda96aa094b4404ad82f6a01d6af78335ed2fb0630f6177c4" exitCode=60
Oct 30 22:00:15.788288 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:15.788191    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" event={"ID":"3955d3d7-0b83-487f-846f-498f29975729","Type":"ContainerDied","Data":"d9026efde5dc4e3eda96aa094b4404ad82f6a01d6af78335ed2fb0630f6177c4"}
Oct 30 22:00:15.788288 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:15.788260    5817 scope.go:122] "RemoveContainer" containerID="8f579a927c2c99ed0446484dd131f5089c3492470f6bbc7de652f6a994cddce1"
Oct 30 22:00:15.788701 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:15.788679    5817 scope.go:122] "RemoveContainer" containerID="d9026efde5dc4e3eda96aa094b4404ad82f6a01d6af78335ed2fb0630f6177c4"
Oct 30 22:00:15.789089 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:15.789061    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7_odh-ai-gateway-infra(3955d3d7-0b83-487f-846f-498f29975729)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" podUID="3955d3d7-0b83-487f-846f-498f29975729"
Oct 30 22:00:15.790756 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:15.790587    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" event={"ID":"245c8d07-a644-42c5-9414-b2d1611f20e5","Type":"ContainerStarted","Data":"0776a99150a1627739e2795521c2f8873ad6d5874ac9f8f0292028ace555dcd3"}
Oct 30 22:00:16.073188 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.073104    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"]
Oct 30 22:00:16.338564 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:16.338471    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:00:16.802341 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.802303    5817 generic.go:356] "Generic (PLEG): container finished" podID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerID="0776a99150a1627739e2795521c2f8873ad6d5874ac9f8f0292028ace555dcd3" exitCode=60
Oct 30 22:00:16.802489 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.802398    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" event={"ID":"245c8d07-a644-42c5-9414-b2d1611f20e5","Type":"ContainerDied","Data":"0776a99150a1627739e2795521c2f8873ad6d5874ac9f8f0292028ace555dcd3"}
Oct 30 22:00:16.802489 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.802470    5817 scope.go:122] "RemoveContainer" containerID="286aff5426a768ca2abeef28781508dbc09acbc48e041ce04b60a4bbcbc8c4d4"
Oct 30 22:00:16.802894 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.802876    5817 scope.go:122] "RemoveContainer" containerID="0776a99150a1627739e2795521c2f8873ad6d5874ac9f8f0292028ace555dcd3"
Oct 30 22:00:16.803303 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:16.803274    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6_odh-ai-gateway-infra(245c8d07-a644-42c5-9414-b2d1611f20e5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5"
Oct 30 22:00:16.804751 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.804733    5817 generic.go:356] "Generic (PLEG): container finished" podID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerID="3a6001aaf488add578535c39bb63dd1e25b6767200579e0cc02fc3835431e5fa" exitCode=60
Oct 30 22:00:16.804823 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.804776    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerDied","Data":"3a6001aaf488add578535c39bb63dd1e25b6767200579e0cc02fc3835431e5fa"}
Oct 30 22:00:16.805130 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.805116    5817 scope.go:122] "RemoveContainer" containerID="3a6001aaf488add578535c39bb63dd1e25b6767200579e0cc02fc3835431e5fa"
Oct 30 22:00:16.805490 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:16.805473    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8_odh-ai-gateway-infra(5fcfc53c-dd7d-45fb-879d-2da556c1b4c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1"
Oct 30 22:00:16.815596 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.815578    5817 scope.go:122] "RemoveContainer" containerID="171f0898644ec5715687b184a5700ab92ceb9ed9e063cecf89b2857a0cfba23f"
Oct 30 22:00:16.838319 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.838284    5817 scope.go:122] "RemoveContainer" containerID="75ea90cbd32cf35f57ae8f18e07043256f47056b1455dacfa6f6ed8f8fe14c23"
Oct 30 22:00:16.846905 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.846876    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" path="/var/lib/kubelet/pods/d4f91b4f-7f3f-492a-aaea-d71076cde8c5/volumes"
Oct 30 22:00:16.912870 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:16.912836    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"]
Oct 30 22:00:17.023317 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.023286    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:17.115716 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.115684    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3955d3d7-0b83-487f-846f-498f29975729-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3955d3d7-0b83-487f-846f-498f29975729-maas-api-service-ca\") pod \"3955d3d7-0b83-487f-846f-498f29975729\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") "
Oct 30 22:00:17.116119 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.115875    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3955d3d7-0b83-487f-846f-498f29975729-kube-api-access-4cht7\" (UniqueName: \"kubernetes.io/projected/3955d3d7-0b83-487f-846f-498f29975729-kube-api-access-4cht7\") pod \"3955d3d7-0b83-487f-846f-498f29975729\" (UID: \"3955d3d7-0b83-487f-846f-498f29975729\") "
Oct 30 22:00:17.116119 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.115992    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3955d3d7-0b83-487f-846f-498f29975729-maas-api-service-ca" pod "3955d3d7-0b83-487f-846f-498f29975729" (UID: "3955d3d7-0b83-487f-846f-498f29975729"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:17.116518 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.116497    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3955d3d7-0b83-487f-846f-498f29975729-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:17.118031 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.118011    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3955d3d7-0b83-487f-846f-498f29975729-kube-api-access-4cht7" pod "3955d3d7-0b83-487f-846f-498f29975729" (UID: "3955d3d7-0b83-487f-846f-498f29975729"). InnerVolumeSpecName "kube-api-access-4cht7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:17.143589 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.143558    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"]
Oct 30 22:00:17.218328 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.218297    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cht7\" (UniqueName: \"kubernetes.io/projected/3955d3d7-0b83-487f-846f-498f29975729-kube-api-access-4cht7\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:17.815275 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.815251    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"
Oct 30 22:00:17.815464 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.815278    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7" event={"ID":"3955d3d7-0b83-487f-846f-498f29975729","Type":"ContainerDied","Data":"fb3aad24f3b161d7c6af308e8db515d403ce91fc30b1d83d9e376a30944d6062"}
Oct 30 22:00:17.815464 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.815395    5817 scope.go:122] "RemoveContainer" containerID="d9026efde5dc4e3eda96aa094b4404ad82f6a01d6af78335ed2fb0630f6177c4"
Oct 30 22:00:17.822852 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.822821    5817 generic.go:356] "Generic (PLEG): container finished" podID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerID="d88811a6ad5c52469850f3643d57990d091fd6d093af7e799b9c873da693ba57" exitCode=60
Oct 30 22:00:17.822968 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.822949    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" event={"ID":"e3f0ee84-2b7d-4258-96ba-d9204cdc5089","Type":"ContainerDied","Data":"d88811a6ad5c52469850f3643d57990d091fd6d093af7e799b9c873da693ba57"}
Oct 30 22:00:17.823459 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.823441    5817 scope.go:122] "RemoveContainer" containerID="d88811a6ad5c52469850f3643d57990d091fd6d093af7e799b9c873da693ba57"
Oct 30 22:00:17.823857 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:17.823833    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9_odh-ai-gateway-infra(e3f0ee84-2b7d-4258-96ba-d9204cdc5089)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089"
Oct 30 22:00:17.836632 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.836606    5817 scope.go:122] "RemoveContainer" containerID="d33b1ca8b6ccb9d83ab9184629fbb568012c69a87759996852385f7d17a5d894"
Oct 30 22:00:17.861583 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:17.861548    5817 scope.go:122] "RemoveContainer" containerID="75ea90cbd32cf35f57ae8f18e07043256f47056b1455dacfa6f6ed8f8fe14c23"
Oct 30 22:00:18.014367 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.011281    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:18.134275 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.134242    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-maas-api-service-ca\") pod \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") "
Oct 30 22:00:18.134733 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.134425    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-kube-api-access-md52j\" (UniqueName: \"kubernetes.io/projected/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-kube-api-access-md52j\") pod \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\" (UID: \"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1\") "
Oct 30 22:00:18.134733 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.134681    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-maas-api-service-ca" pod "5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" (UID: "5fcfc53c-dd7d-45fb-879d-2da556c1b4c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:18.135084 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.135059    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:18.136405 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.136384    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:18.136701 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.136678    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-kube-api-access-md52j" pod "5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" (UID: "5fcfc53c-dd7d-45fb-879d-2da556c1b4c1"). InnerVolumeSpecName "kube-api-access-md52j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:18.199651 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.195807    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"]
Oct 30 22:00:18.203912 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.203640    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-71c0c7-29889960-5wnk7"]
Oct 30 22:00:18.236032 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.236000    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/245c8d07-a644-42c5-9414-b2d1611f20e5-kube-api-access-ldpfn\" (UniqueName: \"kubernetes.io/projected/245c8d07-a644-42c5-9414-b2d1611f20e5-kube-api-access-ldpfn\") pod \"245c8d07-a644-42c5-9414-b2d1611f20e5\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") "
Oct 30 22:00:18.236199 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.236131    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/245c8d07-a644-42c5-9414-b2d1611f20e5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/245c8d07-a644-42c5-9414-b2d1611f20e5-maas-api-service-ca\") pod \"245c8d07-a644-42c5-9414-b2d1611f20e5\" (UID: \"245c8d07-a644-42c5-9414-b2d1611f20e5\") "
Oct 30 22:00:18.236524 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.236500    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245c8d07-a644-42c5-9414-b2d1611f20e5-maas-api-service-ca" pod "245c8d07-a644-42c5-9414-b2d1611f20e5" (UID: "245c8d07-a644-42c5-9414-b2d1611f20e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:18.236741 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.236723    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/245c8d07-a644-42c5-9414-b2d1611f20e5-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:18.236800 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.236747    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md52j\" (UniqueName: \"kubernetes.io/projected/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1-kube-api-access-md52j\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:18.238190 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.238169    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245c8d07-a644-42c5-9414-b2d1611f20e5-kube-api-access-ldpfn" pod "245c8d07-a644-42c5-9414-b2d1611f20e5" (UID: "245c8d07-a644-42c5-9414-b2d1611f20e5"). InnerVolumeSpecName "kube-api-access-ldpfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:18.338100 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.338066    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldpfn\" (UniqueName: \"kubernetes.io/projected/245c8d07-a644-42c5-9414-b2d1611f20e5-kube-api-access-ldpfn\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:18.607400 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.607337    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"]
Oct 30 22:00:18.608126 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608104    5817 cpu_manager.go:414] "containerMap: removing container" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy"
Oct 30 22:00:18.608126 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608123    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608140    5817 cpu_manager.go:414] "containerMap: removing container" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608145    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608155    5817 cpu_manager.go:414] "containerMap: removing container" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608160    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608168    5817 cpu_manager.go:414] "containerMap: removing container" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608173    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608179    5817 cpu_manager.go:414] "containerMap: removing container" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608185    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608212    5817 cpu_manager.go:414] "containerMap: removing container" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608219    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608244    5817 cpu_manager.go:414] "containerMap: removing container" podUID="f7fa220f-5ded-45b4-94b5-400893ffdac5" containerName="payload-pre-processing"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608252    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7fa220f-5ded-45b4-94b5-400893ffdac5" containerName="payload-pre-processing"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608274    5817 cpu_manager.go:414] "containerMap: removing container" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608280    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608296    5817 cpu_manager.go:414] "containerMap: removing container" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608302    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608317    5817 cpu_manager.go:414] "containerMap: removing container" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608323    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608487    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608498    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7fa220f-5ded-45b4-94b5-400893ffdac5" containerName="payload-pre-processing"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608510    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608521    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608531    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608541    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608550    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608558    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608564    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.608844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.608571    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4f91b4f-7f3f-492a-aaea-d71076cde8c5" containerName="istio-proxy"
Oct 30 22:00:18.612150 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.612133    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.678370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.677891    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-28c192\""
Oct 30 22:00:18.687369 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.684106    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"]
Oct 30 22:00:18.697697 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.697649    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-28c192-dockercfg-gfsf7\""
Oct 30 22:00:18.746807 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.746770    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc0e828-5e19-4eed-87df-6ba8727d9c73-plugins-config-volume\") pod \"payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") " pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.747038 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.747020    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cxfn\" (UniqueName: \"kubernetes.io/projected/efc0e828-5e19-4eed-87df-6ba8727d9c73-kube-api-access-6cxfn\") pod \"payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") " pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.822687 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.822657    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"]
Oct 30 22:00:18.823440 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.823406    5817 cpu_manager.go:414] "containerMap: removing container" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.823440 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.823430    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" containerName="cleanup"
Oct 30 22:00:18.823677 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.823509    5817 cpu_manager.go:414] "containerMap: removing container" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.823677 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.823519    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.823677 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.823676    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="3955d3d7-0b83-487f-846f-498f29975729" containerName="cleanup"
Oct 30 22:00:18.823818 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.823734    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" containerName="cleanup"
Oct 30 22:00:18.827335 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.827319    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:18.833261 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.833238    5817 generic.go:356] "Generic (PLEG): container finished" podID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerID="77070964d239fc8a633c706b0803230ebada02f659f8c90a6cf2adb9349c0f46" exitCode=60
Oct 30 22:00:18.833414 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.833303    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" event={"ID":"e80f2967-f96a-46b6-824c-f7bb349a17a7","Type":"ContainerDied","Data":"77070964d239fc8a633c706b0803230ebada02f659f8c90a6cf2adb9349c0f46"}
Oct 30 22:00:18.833414 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.833399    5817 scope.go:122] "RemoveContainer" containerID="d33b1ca8b6ccb9d83ab9184629fbb568012c69a87759996852385f7d17a5d894"
Oct 30 22:00:18.833809 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.833793    5817 scope.go:122] "RemoveContainer" containerID="77070964d239fc8a633c706b0803230ebada02f659f8c90a6cf2adb9349c0f46"
Oct 30 22:00:18.834212 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:18.834175    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk_odh-ai-gateway-infra(e80f2967-f96a-46b6-824c-f7bb349a17a7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7"
Oct 30 22:00:18.835004 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.834980    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6" event={"ID":"245c8d07-a644-42c5-9414-b2d1611f20e5","Type":"ContainerDied","Data":"d573f4b950f8aa135d1a7a337e67be6eecaf5d6e3053d3ef0e7b6a58ee76b5b5"}
Oct 30 22:00:18.835004 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.834994    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"
Oct 30 22:00:18.839221 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.839186    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"
Oct 30 22:00:18.845938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.845912    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3955d3d7-0b83-487f-846f-498f29975729" path="/var/lib/kubelet/pods/3955d3d7-0b83-487f-846f-498f29975729/volumes"
Oct 30 22:00:18.847119 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.847094    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8" event={"ID":"5fcfc53c-dd7d-45fb-879d-2da556c1b4c1","Type":"ContainerDied","Data":"8f55e7e589aeaf8193789c2120e4980aed029379a9bc1fbcd935561586c178e7"}
Oct 30 22:00:18.848493 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.848440    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2szjf\" (UniqueName: \"kubernetes.io/projected/70a9167f-e9f7-4108-876c-22c4784f2c94-kube-api-access-2szjf\") pod \"payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:18.848620 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.848499    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70a9167f-e9f7-4108-876c-22c4784f2c94-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:18.848707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.848638    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc0e828-5e19-4eed-87df-6ba8727d9c73-plugins-config-volume\") pod \"payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") " pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.848707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.848678    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cxfn\" (UniqueName: \"kubernetes.io/projected/efc0e828-5e19-4eed-87df-6ba8727d9c73-kube-api-access-6cxfn\") pod \"payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") " pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.849805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.849784    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc0e828-5e19-4eed-87df-6ba8727d9c73-plugins-config-volume\") pod \"payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") " pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.849918 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.849887    5817 scope.go:122] "RemoveContainer" containerID="0776a99150a1627739e2795521c2f8873ad6d5874ac9f8f0292028ace555dcd3"
Oct 30 22:00:18.860699 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.860678    5817 scope.go:122] "RemoveContainer" containerID="3a6001aaf488add578535c39bb63dd1e25b6767200579e0cc02fc3835431e5fa"
Oct 30 22:00:18.889363 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.889322    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"]
Oct 30 22:00:18.908070 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.908040    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cxfn\" (UniqueName: \"kubernetes.io/projected/efc0e828-5e19-4eed-87df-6ba8727d9c73-kube-api-access-6cxfn\") pod \"payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") " pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.927059 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.927023    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:18.950872 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.950834    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2szjf\" (UniqueName: \"kubernetes.io/projected/70a9167f-e9f7-4108-876c-22c4784f2c94-kube-api-access-2szjf\") pod \"payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:18.951028 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.950908    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70a9167f-e9f7-4108-876c-22c4784f2c94-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:18.951777 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:18.951746    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70a9167f-e9f7-4108-876c-22c4784f2c94-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:19.022212 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.022174    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2szjf\" (UniqueName: \"kubernetes.io/projected/70a9167f-e9f7-4108-876c-22c4784f2c94-kube-api-access-2szjf\") pod \"payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:19.057072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.057038    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"]
Oct 30 22:00:19.066409 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.065415    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"]
Oct 30 22:00:19.071727 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.071699    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-a2dced-29889960-f9rr6"]
Oct 30 22:00:19.144162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.144124    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:19.218103 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.218075    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:19.256265 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.256229    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-kube-api-access-4bkhh\" (UniqueName: \"kubernetes.io/projected/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-kube-api-access-4bkhh\") pod \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") "
Oct 30 22:00:19.256540 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.256517    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-maas-api-service-ca\") pod \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\" (UID: \"e3f0ee84-2b7d-4258-96ba-d9204cdc5089\") "
Oct 30 22:00:19.257078 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.257036    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-maas-api-service-ca" pod "e3f0ee84-2b7d-4258-96ba-d9204cdc5089" (UID: "e3f0ee84-2b7d-4258-96ba-d9204cdc5089"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:19.259790 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.259642    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-kube-api-access-4bkhh" pod "e3f0ee84-2b7d-4258-96ba-d9204cdc5089" (UID: "e3f0ee84-2b7d-4258-96ba-d9204cdc5089"). InnerVolumeSpecName "kube-api-access-4bkhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:19.297900 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.297839    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"]
Oct 30 22:00:19.302806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.302775    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-991335-29889960-7blt8"]
Oct 30 22:00:19.325380 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.325335    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"]
Oct 30 22:00:19.326720 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:19.326694    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc0e828_5e19_4eed_87df_6ba8727d9c73.slice/crio-019c1a77e6783a13ad1898bc3fde4f8d7bcedcdec53633fd7b63d24222b8aa04 WatchSource:0}: Error finding container 019c1a77e6783a13ad1898bc3fde4f8d7bcedcdec53633fd7b63d24222b8aa04: Status 404 returned error can't find the container with id 019c1a77e6783a13ad1898bc3fde4f8d7bcedcdec53633fd7b63d24222b8aa04
Oct 30 22:00:19.358940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.358911    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:19.358940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.358938    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bkhh\" (UniqueName: \"kubernetes.io/projected/e3f0ee84-2b7d-4258-96ba-d9204cdc5089-kube-api-access-4bkhh\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:19.535624 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.535597    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"]
Oct 30 22:00:19.537185 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:19.537152    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a9167f_e9f7_4108_876c_22c4784f2c94.slice/crio-b3e68e7a81966812ab22bd6457e35c770aec54583c9419ebed3bb781111c982c WatchSource:0}: Error finding container b3e68e7a81966812ab22bd6457e35c770aec54583c9419ebed3bb781111c982c: Status 404 returned error can't find the container with id b3e68e7a81966812ab22bd6457e35c770aec54583c9419ebed3bb781111c982c
Oct 30 22:00:19.847950 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.847920    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"
Oct 30 22:00:19.848168 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.847969    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9" event={"ID":"e3f0ee84-2b7d-4258-96ba-d9204cdc5089","Type":"ContainerDied","Data":"0ebf1c9ca0a11bfea6edc6cec4bb20dadf6f5a61edc2cce20fd434404d7abade"}
Oct 30 22:00:19.848168 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.848037    5817 scope.go:122] "RemoveContainer" containerID="d88811a6ad5c52469850f3643d57990d091fd6d093af7e799b9c873da693ba57"
Oct 30 22:00:19.850115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.850094    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq" event={"ID":"efc0e828-5e19-4eed-87df-6ba8727d9c73","Type":"ContainerStarted","Data":"1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e"}
Oct 30 22:00:19.850296 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.850270    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq" event={"ID":"efc0e828-5e19-4eed-87df-6ba8727d9c73","Type":"ContainerStarted","Data":"019c1a77e6783a13ad1898bc3fde4f8d7bcedcdec53633fd7b63d24222b8aa04"}
Oct 30 22:00:19.850478 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.850310    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:19.854830 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.854807    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w" event={"ID":"70a9167f-e9f7-4108-876c-22c4784f2c94","Type":"ContainerStarted","Data":"621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4"}
Oct 30 22:00:19.854951 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.854848    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w" event={"ID":"70a9167f-e9f7-4108-876c-22c4784f2c94","Type":"ContainerStarted","Data":"b3e68e7a81966812ab22bd6457e35c770aec54583c9419ebed3bb781111c982c"}
Oct 30 22:00:19.855011 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:19.854955    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:20.018336 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.018271    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w" podStartSLOduration=2.0182564100000002 podStartE2EDuration="2.01825641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:20.015274465 +0000 UTC m=+2755.302997010" watchObservedRunningTime="2026-10-30 22:00:20.01825641 +0000 UTC m=+2755.305978956"
Oct 30 22:00:20.135213 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.135073    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq" podStartSLOduration=2.135036557 podStartE2EDuration="2.135036557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:20.128158788 +0000 UTC m=+2755.415881338" watchObservedRunningTime="2026-10-30 22:00:20.135036557 +0000 UTC m=+2755.422759104"
Oct 30 22:00:20.198336 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.198298    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"]
Oct 30 22:00:20.238285 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.238237    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"]
Oct 30 22:00:20.257635 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.257605    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-c4e874-29889960-s6jf9"]
Oct 30 22:00:20.392389 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.392295    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:20.477663 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.477625    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e80f2967-f96a-46b6-824c-f7bb349a17a7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80f2967-f96a-46b6-824c-f7bb349a17a7-maas-api-service-ca\") pod \"e80f2967-f96a-46b6-824c-f7bb349a17a7\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") "
Oct 30 22:00:20.477816 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.477745    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e80f2967-f96a-46b6-824c-f7bb349a17a7-kube-api-access-cb7kc\" (UniqueName: \"kubernetes.io/projected/e80f2967-f96a-46b6-824c-f7bb349a17a7-kube-api-access-cb7kc\") pod \"e80f2967-f96a-46b6-824c-f7bb349a17a7\" (UID: \"e80f2967-f96a-46b6-824c-f7bb349a17a7\") "
Oct 30 22:00:20.478057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.478033    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e80f2967-f96a-46b6-824c-f7bb349a17a7-maas-api-service-ca" pod "e80f2967-f96a-46b6-824c-f7bb349a17a7" (UID: "e80f2967-f96a-46b6-824c-f7bb349a17a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:20.478476 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.478453    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80f2967-f96a-46b6-824c-f7bb349a17a7-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:20.480055 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.480024    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e80f2967-f96a-46b6-824c-f7bb349a17a7-kube-api-access-cb7kc" pod "e80f2967-f96a-46b6-824c-f7bb349a17a7" (UID: "e80f2967-f96a-46b6-824c-f7bb349a17a7"). InnerVolumeSpecName "kube-api-access-cb7kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:20.580292 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.580258    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb7kc\" (UniqueName: \"kubernetes.io/projected/e80f2967-f96a-46b6-824c-f7bb349a17a7-kube-api-access-cb7kc\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:20.670313 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.670222    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"]
Oct 30 22:00:20.671121 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671098    5817 cpu_manager.go:414] "containerMap: removing container" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671121 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671120    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671152    5817 cpu_manager.go:414] "containerMap: removing container" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671158    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671172    5817 cpu_manager.go:414] "containerMap: removing container" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671176    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671183    5817 cpu_manager.go:414] "containerMap: removing container" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671188    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671199    5817 cpu_manager.go:414] "containerMap: removing container" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671203    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671231    5817 cpu_manager.go:414] "containerMap: removing container" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.671252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671236    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.671615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671357    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671377    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671388    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.671615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671395    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" containerName="cleanup"
Oct 30 22:00:20.671615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.671410    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:20.675094 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.675070    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.743132 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.742907    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"]
Oct 30 22:00:20.783504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783462    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.783713 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783532    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.783713 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783657    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5j24\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-kube-api-access-k5j24\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.783842 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783760    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18e6ea2-6992-427d-8c0f-6244c76052d6-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.783893 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783838    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.783932 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783906    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.784043 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783932    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.784043 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.783966    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.784199 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.784182    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.845963 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.845923    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="245c8d07-a644-42c5-9414-b2d1611f20e5" path="/var/lib/kubelet/pods/245c8d07-a644-42c5-9414-b2d1611f20e5/volumes"
Oct 30 22:00:20.846484 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.846466    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fcfc53c-dd7d-45fb-879d-2da556c1b4c1" path="/var/lib/kubelet/pods/5fcfc53c-dd7d-45fb-879d-2da556c1b4c1/volumes"
Oct 30 22:00:20.846932 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.846915    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3f0ee84-2b7d-4258-96ba-d9204cdc5089" path="/var/lib/kubelet/pods/e3f0ee84-2b7d-4258-96ba-d9204cdc5089/volumes"
Oct 30 22:00:20.863911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.863889    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"
Oct 30 22:00:20.863911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.863896    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk" event={"ID":"e80f2967-f96a-46b6-824c-f7bb349a17a7","Type":"ContainerDied","Data":"097b6849242924e7f464819aef9f53f0e101ad05d233934ff6ee48a8d1466654"}
Oct 30 22:00:20.864119 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.863959    5817 scope.go:122] "RemoveContainer" containerID="77070964d239fc8a633c706b0803230ebada02f659f8c90a6cf2adb9349c0f46"
Oct 30 22:00:20.885465 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.885435    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.885685 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.885664    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.885814 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.885793    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.885925 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.885836    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.885925 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.885899    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5j24\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-kube-api-access-k5j24\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886034 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.885994    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18e6ea2-6992-427d-8c0f-6244c76052d6-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886088 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886026    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886088 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886065    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886194 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886102    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886194 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886125    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886194 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886178    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886537 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886503    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886717 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886691    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18e6ea2-6992-427d-8c0f-6244c76052d6-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.886979 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.886924    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.888188 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.888168    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.888887 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.888865    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.945553 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.945479    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.945799 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.945752    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5j24\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-kube-api-access-k5j24\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:20.989412 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:20.989383    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:21.020958 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.020928    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"]
Oct 30 22:00:21.027755 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.027724    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-ed76b8-29889960-kt2lk"]
Oct 30 22:00:21.431301 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.431253    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"]
Oct 30 22:00:21.433227 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:21.433196    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18e6ea2_6992_427d_8c0f_6244c76052d6.slice/crio-4762d213b5999565e524d2f02b9c1a84e5411a8ad2ba90a07dd573588eaa4a3b WatchSource:0}: Error finding container 4762d213b5999565e524d2f02b9c1a84e5411a8ad2ba90a07dd573588eaa4a3b: Status 404 returned error can't find the container with id 4762d213b5999565e524d2f02b9c1a84e5411a8ad2ba90a07dd573588eaa4a3b
Oct 30 22:00:21.437055 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.437010    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:21.437179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.437106    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:21.437179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.437152    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:21.875643 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.875599    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" event={"ID":"a18e6ea2-6992-427d-8c0f-6244c76052d6","Type":"ContainerStarted","Data":"28ca5ed569ca1ad44b9319d31535c33152d82e307fda7d6ecb66ab1cd4411de4"}
Oct 30 22:00:21.875877 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.875671    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" event={"ID":"a18e6ea2-6992-427d-8c0f-6244c76052d6","Type":"ContainerStarted","Data":"4762d213b5999565e524d2f02b9c1a84e5411a8ad2ba90a07dd573588eaa4a3b"}
Oct 30 22:00:21.990503 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:21.990471    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:22.023967 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:22.023917    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podStartSLOduration=2.023902712 podStartE2EDuration="2.023902712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:22.020531735 +0000 UTC m=+2757.308254281" watchObservedRunningTime="2026-10-30 22:00:22.023902712 +0000 UTC m=+2757.311625257"
Oct 30 22:00:22.846789 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:22.846749    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" path="/var/lib/kubelet/pods/e80f2967-f96a-46b6-824c-f7bb349a17a7/volumes"
Oct 30 22:00:23.011370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:23.008741    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:00:23.011370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:23.008816    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 22:00:24.008370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:24.007797    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:00:24.008370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:24.007868    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 22:00:25.011506 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:25.011456    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:00:25.012178 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:25.012143    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 22:00:26.005373 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:25.992727    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:00:26.005373 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:25.992788    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 22:00:27.013823 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:27.013784    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:00:27.014472 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:27.014427    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 22:00:27.990264 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:27.990210    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 22:00:27.990498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:27.990283    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 22:00:28.994923 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:28.994865    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:28.995392 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:28.995138    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:28.995889 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:28.995869    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:00:30.869115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:30.869087    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:00:30.869586 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:30.869159    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:00:33.756894 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:33.756843    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"]
Oct 30 22:00:33.757676 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:33.757212    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2" podUID="5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" containerName="maas-api" containerID="cri-o://dfb610cfc413e6b07bb8e136e6179677702084c781f0d17856f6b055f82ce982" gracePeriod=30
Oct 30 22:00:33.998315 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:33.998278    5817 generic.go:356] "Generic (PLEG): container finished" podID="5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" containerID="dfb610cfc413e6b07bb8e136e6179677702084c781f0d17856f6b055f82ce982" exitCode=0
Oct 30 22:00:33.998512 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:33.998389    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2" event={"ID":"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09","Type":"ContainerDied","Data":"dfb610cfc413e6b07bb8e136e6179677702084c781f0d17856f6b055f82ce982"}
Oct 30 22:00:34.040258 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.040223    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 22:00:34.120785 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.120748    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls\") pod \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") "
Oct 30 22:00:34.120956 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.120795    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-kube-api-access-zv26c\" (UniqueName: \"kubernetes.io/projected/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-kube-api-access-zv26c\") pod \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") "
Oct 30 22:00:34.120956 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.120929    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-openshift-service-ca\") pod \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") "
Oct 30 22:00:34.121036 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.120976    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-metrics-certs\") pod \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\" (UID: \"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09\") "
Oct 30 22:00:34.121295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.121269    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-openshift-service-ca" pod "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" (UID: "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:34.121697 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.121676    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:34.123026 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.123002    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-kube-api-access-zv26c" pod "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" (UID: "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09"). InnerVolumeSpecName "kube-api-access-zv26c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:34.123183 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.123101    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-metrics-certs" pod "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" (UID: "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:34.123183 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.123133    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls" pod "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" (UID: "5c2bb1d9-1a99-4812-9aff-50f0ece9ad09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:34.223265 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.223223    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:34.223265 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.223259    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv26c\" (UniqueName: \"kubernetes.io/projected/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-kube-api-access-zv26c\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:34.223265 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:34.223270    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:35.006785 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.006749    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"
Oct 30 22:00:35.007239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.006783    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2" event={"ID":"5c2bb1d9-1a99-4812-9aff-50f0ece9ad09","Type":"ContainerDied","Data":"b15aebb7c2aa45045f3ab785cc94dcd23a80255697d18415a7b29eda83654ef6"}
Oct 30 22:00:35.007239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.006850    5817 scope.go:122] "RemoveContainer" containerID="dfb610cfc413e6b07bb8e136e6179677702084c781f0d17856f6b055f82ce982"
Oct 30 22:00:35.188398 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.188302    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"]
Oct 30 22:00:35.218635 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.218603    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ed6135-7dcbcfd6f4-5tml2"]
Oct 30 22:00:35.504662 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.504623    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-11379-4667b9"]
Oct 30 22:00:35.505541 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.505514    5817 cpu_manager.go:414] "containerMap: removing container" podUID="5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" containerName="maas-api"
Oct 30 22:00:35.505541 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.505541    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" containerName="maas-api"
Oct 30 22:00:35.505712 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.505657    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" containerName="maas-api"
Oct 30 22:00:35.505712 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.505677    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="e80f2967-f96a-46b6-824c-f7bb349a17a7" containerName="cleanup"
Oct 30 22:00:35.510614 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.510594    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:35.541214 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.541184    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndnxq\" (UniqueName: \"kubernetes.io/projected/dc6de55c-1998-478b-b65c-9b59c3b30a13-kube-api-access-ndnxq\") pod \"test-curl-11379-4667b9\" (UID: \"dc6de55c-1998-478b-b65c-9b59c3b30a13\") " pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:35.560388 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.560365    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-sxztf\""
Oct 30 22:00:35.601592 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.601561    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-11379-4667b9"]
Oct 30 22:00:35.642600 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.642565    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndnxq\" (UniqueName: \"kubernetes.io/projected/dc6de55c-1998-478b-b65c-9b59c3b30a13-kube-api-access-ndnxq\") pod \"test-curl-11379-4667b9\" (UID: \"dc6de55c-1998-478b-b65c-9b59c3b30a13\") " pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:35.700843 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.700807    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndnxq\" (UniqueName: \"kubernetes.io/projected/dc6de55c-1998-478b-b65c-9b59c3b30a13-kube-api-access-ndnxq\") pod \"test-curl-11379-4667b9\" (UID: \"dc6de55c-1998-478b-b65c-9b59c3b30a13\") " pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:35.823403 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.823258    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:35.825606 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.825556    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"]
Oct 30 22:00:35.830399 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.830373    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.846271 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846241    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.846456 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846292    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.846677 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846648    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.846725 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846697    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.846775 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846758    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.846904 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846887    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.846976 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846929    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.847030 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.846983    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ws2m\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-kube-api-access-7ws2m\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.847086 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.847034    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.889810 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.889522    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"]
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948197    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948254    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948426    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948456    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948492    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948633    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948685    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948743    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ws2m\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-kube-api-access-7ws2m\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.948796    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.949264    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-data\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.949655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.949543    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.950414 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.949780    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-certs\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.950488 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.950435    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istiod-ca-cert\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.952299 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.952253    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-credential-socket\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.972294 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.972260    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-podinfo\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:35.973335 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:35.973312    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-envoy\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:36.008892 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.008863    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-token\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:36.010611 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.010588    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ws2m\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-kube-api-access-7ws2m\") pod \"e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") " pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:36.179988 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.179891    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:36.209701 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:36.209668    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc6de55c_1998_478b_b65c_9b59c3b30a13.slice/crio-2e0bf5ec72fb2f964063e7cd5135778eeea31ae04c13125d3ea147174e637b45 WatchSource:0}: Error finding container 2e0bf5ec72fb2f964063e7cd5135778eeea31ae04c13125d3ea147174e637b45: Status 404 returned error can't find the container with id 2e0bf5ec72fb2f964063e7cd5135778eeea31ae04c13125d3ea147174e637b45
Oct 30 22:00:36.209701 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.209676    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-11379-4667b9"]
Oct 30 22:00:36.211902 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.211885    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:00:36.614215 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.614172    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"]
Oct 30 22:00:36.616925 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:00:36.616895    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11adea8b_7c70_46c4_84cd_d3ed86fcd343.slice/crio-f88880fa446e339694ede4bc376712b5ce39939873ce432133f4cc3074ba0928 WatchSource:0}: Error finding container f88880fa446e339694ede4bc376712b5ce39939873ce432133f4cc3074ba0928: Status 404 returned error can't find the container with id f88880fa446e339694ede4bc376712b5ce39939873ce432133f4cc3074ba0928
Oct 30 22:00:36.619732 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.619696    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:36.619809 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.619769    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:36.619809 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.619799    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:00:36.847922 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:36.847884    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c2bb1d9-1a99-4812-9aff-50f0ece9ad09" path="/var/lib/kubelet/pods/5c2bb1d9-1a99-4812-9aff-50f0ece9ad09/volumes"
Oct 30 22:00:37.023012 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:37.022975    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" event={"ID":"11adea8b-7c70-46c4-84cd-d3ed86fcd343","Type":"ContainerStarted","Data":"6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1"}
Oct 30 22:00:37.023463 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:37.023043    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" event={"ID":"11adea8b-7c70-46c4-84cd-d3ed86fcd343","Type":"ContainerStarted","Data":"f88880fa446e339694ede4bc376712b5ce39939873ce432133f4cc3074ba0928"}
Oct 30 22:00:37.024277 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:37.024254    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-11379-4667b9" event={"ID":"dc6de55c-1998-478b-b65c-9b59c3b30a13","Type":"ContainerStarted","Data":"2e0bf5ec72fb2f964063e7cd5135778eeea31ae04c13125d3ea147174e637b45"}
Oct 30 22:00:37.139437 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:37.139381    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podStartSLOduration=2.139333645 podStartE2EDuration="2.139333645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:37.137661647 +0000 UTC m=+2772.425384193" watchObservedRunningTime="2026-10-30 22:00:37.139333645 +0000 UTC m=+2772.427056230"
Oct 30 22:00:37.180661 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:37.180614    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:37.182149 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:37.182124    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:00:37.182281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:37.182172    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 22:00:38.180322 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:38.180285    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:00:38.180781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:38.180341    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 22:00:39.181086 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:39.181048    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:00:39.181606 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:39.181109    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 22:00:40.181048 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:40.181008    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:00:40.181207 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:40.181081    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 22:00:41.119924 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:41.119879    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-11379-4667b9" event={"ID":"dc6de55c-1998-478b-b65c-9b59c3b30a13","Type":"ContainerStarted","Data":"c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87"}
Oct 30 22:00:41.180755 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:41.180716    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:00:41.180931 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:41.180770    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 22:00:41.232306 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:41.232244    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-11379-4667b9" podStartSLOduration=2.095345327 podStartE2EDuration="6.232228759s" totalImagesPullingTime="4.136883432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:41.230478394 +0000 UTC m=+2776.518200940" watchObservedRunningTime="2026-10-30 22:00:41.232228759 +0000 UTC m=+2776.519951304"
Oct 30 22:00:42.180326 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:42.180283    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:00:42.180567 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:42.180391    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.131:15021/healthz/ready\": dial tcp 10.134.0.131:15021: connect: connection refused"
Oct 30 22:00:42.764236 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:42.764197    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-11379-4667b9"]
Oct 30 22:00:42.771526 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:42.771493    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-11379-4667b9"]
Oct 30 22:00:43.135237 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:43.135131    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-11379-4667b9" podUID="dc6de55c-1998-478b-b65c-9b59c3b30a13" containerName="test-curl-11379-4667b9" containerID="cri-o://c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87" gracePeriod=2
Oct 30 22:00:43.251020 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:43.250973    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:43.251727 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:43.251664    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:43.253650 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:43.253549    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:00:43.340258 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:43.340227    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"]
Oct 30 22:00:43.340541 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:43.340512    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy" containerID="cri-o://e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96" gracePeriod=30
Oct 30 22:00:44.930728 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:44.930687    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"]
Oct 30 22:00:44.931124 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:44.930950    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" containerID="cri-o://ddad09c07560489410506faab96eda27b39ba978817e79dbdc322bfb87d7b694" gracePeriod=30
Oct 30 22:00:45.389565 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:45.389535    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:45.441505 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:45.441436    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc6de55c-1998-478b-b65c-9b59c3b30a13-kube-api-access-ndnxq\" (UniqueName: \"kubernetes.io/projected/dc6de55c-1998-478b-b65c-9b59c3b30a13-kube-api-access-ndnxq\") pod \"dc6de55c-1998-478b-b65c-9b59c3b30a13\" (UID: \"dc6de55c-1998-478b-b65c-9b59c3b30a13\") "
Oct 30 22:00:45.443654 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:45.443623    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc6de55c-1998-478b-b65c-9b59c3b30a13-kube-api-access-ndnxq" pod "dc6de55c-1998-478b-b65c-9b59c3b30a13" (UID: "dc6de55c-1998-478b-b65c-9b59c3b30a13"). InnerVolumeSpecName "kube-api-access-ndnxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:45.543029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:45.542992    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndnxq\" (UniqueName: \"kubernetes.io/projected/dc6de55c-1998-478b-b65c-9b59c3b30a13-kube-api-access-ndnxq\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:46.156790 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:46.156747    5817 generic.go:356] "Generic (PLEG): container finished" podID="dc6de55c-1998-478b-b65c-9b59c3b30a13" containerID="c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87" exitCode=137
Oct 30 22:00:46.157232 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:46.156808    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:46.157232 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:46.156845    5817 scope.go:122] "RemoveContainer" containerID="c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87"
Oct 30 22:00:46.170484 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:46.170455    5817 scope.go:122] "RemoveContainer" containerID="c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87"
Oct 30 22:00:46.170773 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:46.170750    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87\": container with ID starting with c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87 not found: ID does not exist" containerID="c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87"
Oct 30 22:00:46.170843 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:46.170784    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87"} err="failed to get container status \"c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87\": rpc error: code = NotFound desc = could not find container \"c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87\": container with ID starting with c1d5fc5cd7288581dcf0e8e0cb719d2c81dc29f3aa737cc9bb88f5a1e32c3f87 not found: ID does not exist"
Oct 30 22:00:46.217586 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:46.217549    5817 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-11379-4667b9\" is forbidden: User \"system:node:ip-10-0-142-25.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-25.us-west-2.compute.internal' and this object" podUID="dc6de55c-1998-478b-b65c-9b59c3b30a13" pod="opendatahub/test-curl-11379-4667b9"
Oct 30 22:00:46.339564 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:46.339523    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:00:46.844909 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:46.844880    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc6de55c-1998-478b-b65c-9b59c3b30a13" path="/var/lib/kubelet/pods/dc6de55c-1998-478b-b65c-9b59c3b30a13/volumes"
Oct 30 22:00:48.636901 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.636874    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 22:00:48.797607 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797579    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-credential-socket\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.797766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797657    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-envoy\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.797766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797725    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-data\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.797766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797760    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-socket\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.797948 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797794    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istiod-ca-cert\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.797948 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797853    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-kube-api-access-bhg2x\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-kube-api-access-bhg2x\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.797948 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797856    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-credential-socket" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:48.797948 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797882    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-podinfo\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.797948 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797943    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-certs\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.798220 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.797991    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-token\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-token\") pod \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\" (UID: \"f7f80f7d-26be-43b8-b355-9c7e8a9924d7\") "
Oct 30 22:00:48.798220 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798096    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-data" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:48.798220 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798208    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-certs" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:48.798438 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798235    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istiod-ca-cert" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:48.798438 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798278    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-socket" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:48.798777 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798754    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.798833 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798783    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.798833 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798800    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.798833 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798817    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.798927 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.798832    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.800044 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.800025    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-podinfo" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:00:48.800162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.800146    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-kube-api-access-bhg2x" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "kube-api-access-bhg2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:48.800383 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.800315    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-envoy" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:48.800502 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.800448    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-token" pod "f7f80f7d-26be-43b8-b355-9c7e8a9924d7" (UID: "f7f80f7d-26be-43b8-b355-9c7e8a9924d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:48.900170 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.900079    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.900170 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.900111    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhg2x\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-kube-api-access-bhg2x\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.900170 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.900120    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:48.900170 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:48.900130    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7f80f7d-26be-43b8-b355-9c7e8a9924d7-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:49.181124 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.181018    5817 generic.go:356] "Generic (PLEG): container finished" podID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerID="e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96" exitCode=0
Oct 30 22:00:49.181124 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.181073    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"
Oct 30 22:00:49.181124 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.181107    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" event={"ID":"f7f80f7d-26be-43b8-b355-9c7e8a9924d7","Type":"ContainerDied","Data":"e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96"}
Oct 30 22:00:49.181374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.181163    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk" event={"ID":"f7f80f7d-26be-43b8-b355-9c7e8a9924d7","Type":"ContainerDied","Data":"50360112bd9d954a7744eae3546d9bd0fffb68a71b6d013e4860f153ffe4514e"}
Oct 30 22:00:49.181374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.181189    5817 scope.go:122] "RemoveContainer" containerID="e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96"
Oct 30 22:00:49.191914 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.191898    5817 scope.go:122] "RemoveContainer" containerID="b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb"
Oct 30 22:00:49.201209 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.201192    5817 scope.go:122] "RemoveContainer" containerID="e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96"
Oct 30 22:00:49.201511 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:49.201492    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96\": container with ID starting with e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96 not found: ID does not exist" containerID="e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96"
Oct 30 22:00:49.201572 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.201519    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96"} err="failed to get container status \"e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96\": rpc error: code = NotFound desc = could not find container \"e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96\": container with ID starting with e3518f673bf585452380620cbf2e002c3ae16e396ddd1b71c2c81d6a0c76fa96 not found: ID does not exist"
Oct 30 22:00:49.201572 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.201536    5817 scope.go:122] "RemoveContainer" containerID="b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb"
Oct 30 22:00:49.201808 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:49.201794    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb\": container with ID starting with b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb not found: ID does not exist" containerID="b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb"
Oct 30 22:00:49.201849 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.201810    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb"} err="failed to get container status \"b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb\": rpc error: code = NotFound desc = could not find container \"b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb\": container with ID starting with b3d3aeb678bd81af09ea600fda1c4bb837987cfc45aa068d9e6ba5cf132cbbbb not found: ID does not exist"
Oct 30 22:00:49.320868 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.320832    5817 patch_prober.go:29] interesting pod/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:00:49.321029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.320890    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:00:49.364164 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.364113    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"]
Oct 30 22:00:49.375588 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:49.375555    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-554f46-openshift-default-79c6b6dcf7-tdfbk"]
Oct 30 22:00:50.191042 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.191013    5817 generic.go:356] "Generic (PLEG): container finished" podID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerID="ddad09c07560489410506faab96eda27b39ba978817e79dbdc322bfb87d7b694" exitCode=0
Oct 30 22:00:50.191471 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.191081    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" event={"ID":"0a0ddf30-bcf7-4449-8871-9245159c0b09","Type":"ContainerDied","Data":"ddad09c07560489410506faab96eda27b39ba978817e79dbdc322bfb87d7b694"}
Oct 30 22:00:50.191471 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.191143    5817 scope.go:122] "RemoveContainer" containerID="429d30769691934013a89accc134839e359b067a75bd1d05be429df09aa78cc5"
Oct 30 22:00:50.217451 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.217431    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 22:00:50.239776 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.239743    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"]
Oct 30 22:00:50.239951 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.239930    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj" podUID="85da0cc8-f192-4bfa-8a98-1a5740da53c1" containerName="maas-api" containerID="cri-o://d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30" gracePeriod=30
Oct 30 22:00:50.321223 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321182    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-data\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321434 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321260    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-kube-api-access-npwr4\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-kube-api-access-npwr4\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321434 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321375    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-podinfo\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321579 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321463    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-token\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-token\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321579 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321498    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-data" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:50.321579 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321535    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-socket\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321579 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321565    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a0ddf30-bcf7-4449-8871-9245159c0b09-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0ddf30-bcf7-4449-8871-9245159c0b09-istiod-ca-cert\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321742 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321609    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-envoy\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321742 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321642    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-credential-socket\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.321837 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.321756    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-certs\") pod \"0a0ddf30-bcf7-4449-8871-9245159c0b09\" (UID: \"0a0ddf30-bcf7-4449-8871-9245159c0b09\") "
Oct 30 22:00:50.322539 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.322104    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-credential-socket" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:50.322539 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.322380    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-certs" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:50.322539 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.322481    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.322539 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.322505    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.322875 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.322851    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a0ddf30-bcf7-4449-8871-9245159c0b09-istiod-ca-cert" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:50.322950 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.322913    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-socket" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:50.325413 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.324587    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-envoy" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:00:50.325413 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.325310    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-kube-api-access-npwr4" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "kube-api-access-npwr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:50.326151 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.326128    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-podinfo" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:00:50.326414 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.326392    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-token" pod "0a0ddf30-bcf7-4449-8871-9245159c0b09" (UID: "0a0ddf30-bcf7-4449-8871-9245159c0b09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:50.423187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.423147    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"]
Oct 30 22:00:50.424410 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.424335    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.424532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.424417    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npwr4\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-kube-api-access-npwr4\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.424532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.424434    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.424532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.424448    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.424532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.424462    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.424532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.424476    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a0ddf30-bcf7-4449-8871-9245159c0b09-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.424532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.424490    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a0ddf30-bcf7-4449-8871-9245159c0b09-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.454614 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.453709    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"]
Oct 30 22:00:50.482041 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.482009    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"]
Oct 30 22:00:50.509432 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.509406    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 22:00:50.626463 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.626429    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85da0cc8-f192-4bfa-8a98-1a5740da53c1-kube-api-access-rmkn7\" (UniqueName: \"kubernetes.io/projected/85da0cc8-f192-4bfa-8a98-1a5740da53c1-kube-api-access-rmkn7\") pod \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") "
Oct 30 22:00:50.626612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.626491    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-metrics-certs\") pod \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") "
Oct 30 22:00:50.626612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.626570    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls\") pod \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") "
Oct 30 22:00:50.626612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.626594    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85da0cc8-f192-4bfa-8a98-1a5740da53c1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85da0cc8-f192-4bfa-8a98-1a5740da53c1-openshift-service-ca\") pod \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\" (UID: \"85da0cc8-f192-4bfa-8a98-1a5740da53c1\") "
Oct 30 22:00:50.627005 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.626969    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85da0cc8-f192-4bfa-8a98-1a5740da53c1-openshift-service-ca" pod "85da0cc8-f192-4bfa-8a98-1a5740da53c1" (UID: "85da0cc8-f192-4bfa-8a98-1a5740da53c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:50.627284 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.627262    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85da0cc8-f192-4bfa-8a98-1a5740da53c1-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.628789 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.628760    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-metrics-certs" pod "85da0cc8-f192-4bfa-8a98-1a5740da53c1" (UID: "85da0cc8-f192-4bfa-8a98-1a5740da53c1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:50.628789 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.628777    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls" pod "85da0cc8-f192-4bfa-8a98-1a5740da53c1" (UID: "85da0cc8-f192-4bfa-8a98-1a5740da53c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:00:50.628910 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.628812    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85da0cc8-f192-4bfa-8a98-1a5740da53c1-kube-api-access-rmkn7" pod "85da0cc8-f192-4bfa-8a98-1a5740da53c1" (UID: "85da0cc8-f192-4bfa-8a98-1a5740da53c1"). InnerVolumeSpecName "kube-api-access-rmkn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:50.729283 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.729201    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmkn7\" (UniqueName: \"kubernetes.io/projected/85da0cc8-f192-4bfa-8a98-1a5740da53c1-kube-api-access-rmkn7\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.729283 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.729231    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.729283 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.729243    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85da0cc8-f192-4bfa-8a98-1a5740da53c1-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:00:50.844092 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:50.844057    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" path="/var/lib/kubelet/pods/f7f80f7d-26be-43b8-b355-9c7e8a9924d7/volumes"
Oct 30 22:00:51.199732 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.199651    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"
Oct 30 22:00:51.199732 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.199683    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz" event={"ID":"0a0ddf30-bcf7-4449-8871-9245159c0b09","Type":"ContainerDied","Data":"c18aa060d0a0907e86a02fdc77736b0fe13bc819ea22912072e7117544c1ddda"}
Oct 30 22:00:51.200218 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.199744    5817 scope.go:122] "RemoveContainer" containerID="ddad09c07560489410506faab96eda27b39ba978817e79dbdc322bfb87d7b694"
Oct 30 22:00:51.201144 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.201123    5817 generic.go:356] "Generic (PLEG): container finished" podID="85da0cc8-f192-4bfa-8a98-1a5740da53c1" containerID="d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30" exitCode=0
Oct 30 22:00:51.201233 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.201158    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj" event={"ID":"85da0cc8-f192-4bfa-8a98-1a5740da53c1","Type":"ContainerDied","Data":"d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30"}
Oct 30 22:00:51.201233 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.201177    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"
Oct 30 22:00:51.201233 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.201215    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj" event={"ID":"85da0cc8-f192-4bfa-8a98-1a5740da53c1","Type":"ContainerDied","Data":"85ae8d8d9e9f4c2ccd4e36237b380514619fac567f314374687cddea7a110551"}
Oct 30 22:00:51.210237 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.210217    5817 scope.go:122] "RemoveContainer" containerID="d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30"
Oct 30 22:00:51.219373 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.219341    5817 scope.go:122] "RemoveContainer" containerID="d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30"
Oct 30 22:00:51.219674 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:00:51.219651    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30\": container with ID starting with d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30 not found: ID does not exist" containerID="d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30"
Oct 30 22:00:51.219766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.219679    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30"} err="failed to get container status \"d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30\": rpc error: code = NotFound desc = could not find container \"d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30\": container with ID starting with d375b54c275569a54977a65ad66df270c9a2c393fbab7a1279e16a9071301a30 not found: ID does not exist"
Oct 30 22:00:51.435967 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.435926    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"]
Oct 30 22:00:51.440068 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.440037    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ed6135-openshift-default-86cf75d87f-58drz"]
Oct 30 22:00:51.551530 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.551468    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"]
Oct 30 22:00:51.555003 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:51.554976    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2df7b6-b8c7f8675-bp4cj"]
Oct 30 22:00:52.847999 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:52.847955    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" path="/var/lib/kubelet/pods/0a0ddf30-bcf7-4449-8871-9245159c0b09/volumes"
Oct 30 22:00:52.848699 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:52.848677    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85da0cc8-f192-4bfa-8a98-1a5740da53c1" path="/var/lib/kubelet/pods/85da0cc8-f192-4bfa-8a98-1a5740da53c1/volumes"
Oct 30 22:00:57.698764 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:57.698727    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"]
Oct 30 22:00:57.807638 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:00:57.807603    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"]
Oct 30 22:01:02.699402 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:02.699287    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" podUID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" containerName="payload-processing" containerID="cri-o://49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684" gracePeriod=25
Oct 30 22:01:02.808513 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:02.808458    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" podUID="5b8250db-6367-4c18-8898-c0561e4364e1" containerName="payload-pre-processing" containerID="cri-o://e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef" gracePeriod=25
Oct 30 22:01:05.452973 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.452903    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main" containerID="cri-o://800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928" gracePeriod=45
Oct 30 22:01:05.478045 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.477995    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" containerID="cri-o://6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d" gracePeriod=45
Oct 30 22:01:05.504342 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.504300    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" containerID="cri-o://c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d" gracePeriod=45
Oct 30 22:01:05.579768 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.579729    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": dial tcp 10.134.0.56:8000: connect: connection refused"
Oct 30 22:01:05.708426 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.708055    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": dial tcp 10.134.0.53:8000: connect: connection refused"
Oct 30 22:01:05.947170 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.947140    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 22:01:05.961197 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.961133    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 22:01:05.965533 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:05.965490    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 22:01:06.024052 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024021    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-dshm\") pod \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") "
Oct 30 22:01:06.024237 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024086    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-model-cache\") pod \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") "
Oct 30 22:01:06.024237 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024203    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tmp-dir\") pod \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") "
Oct 30 22:01:06.024313 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024238    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-home\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-home\") pod \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") "
Oct 30 22:01:06.024313 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024289    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tls-certs\") pod \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") "
Oct 30 22:01:06.024431 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024362    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-model-cache" pod "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" (UID: "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.024530 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024512    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-kube-api-access-266wt\" (UniqueName: \"kubernetes.io/projected/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-kube-api-access-266wt\") pod \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\" (UID: \"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530\") "
Oct 30 22:01:06.024595 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024514    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tmp-dir" pod "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" (UID: "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.024595 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.024528    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-home" pod "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" (UID: "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.025028 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.025007    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.025162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.025048    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.025162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.025057    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.026519 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.026499    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-kube-api-access-266wt" pod "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" (UID: "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530"). InnerVolumeSpecName "kube-api-access-266wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:06.026611 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.026519    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-dshm" pod "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" (UID: "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.026611 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.026546    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tls-certs" pod "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" (UID: "d9c6d767-7ebe-46d5-8de8-fc46d3eaa530"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:01:06.126332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126298    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c05da20-a98a-4b76-9db6-44d92303fe12-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c05da20-a98a-4b76-9db6-44d92303fe12-tls-certs\") pod \"6c05da20-a98a-4b76-9db6-44d92303fe12\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") "
Oct 30 22:01:06.126540 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126487    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-home\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-home\") pod \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") "
Oct 30 22:01:06.126540 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126519    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-dshm\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-dshm\") pod \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") "
Oct 30 22:01:06.126643 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126555    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-model-cache\") pod \"6c05da20-a98a-4b76-9db6-44d92303fe12\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") "
Oct 30 22:01:06.126643 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126637    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tls-certs\" (UniqueName: \"kubernetes.io/secret/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tls-certs\") pod \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") "
Oct 30 22:01:06.126757 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126682    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tmp-dir\") pod \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") "
Oct 30 22:01:06.126757 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126719    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c05da20-a98a-4b76-9db6-44d92303fe12-kube-api-access-gh4x7\" (UniqueName: \"kubernetes.io/projected/6c05da20-a98a-4b76-9db6-44d92303fe12-kube-api-access-gh4x7\") pod \"6c05da20-a98a-4b76-9db6-44d92303fe12\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") "
Oct 30 22:01:06.126757 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126740    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-home" pod "410f53f3-a093-4f1f-8cf6-fb7cd5eda739" (UID: "410f53f3-a093-4f1f-8cf6-fb7cd5eda739"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.126757 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126752    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-home\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-home\") pod \"6c05da20-a98a-4b76-9db6-44d92303fe12\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") "
Oct 30 22:01:06.126964 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126868    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-dshm\") pod \"6c05da20-a98a-4b76-9db6-44d92303fe12\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") "
Oct 30 22:01:06.126964 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126895    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-tmp-dir\") pod \"6c05da20-a98a-4b76-9db6-44d92303fe12\" (UID: \"6c05da20-a98a-4b76-9db6-44d92303fe12\") "
Oct 30 22:01:06.126964 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126919    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-model-cache" pod "6c05da20-a98a-4b76-9db6-44d92303fe12" (UID: "6c05da20-a98a-4b76-9db6-44d92303fe12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.127123 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126947    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-kube-api-access-hjjkv\" (UniqueName: \"kubernetes.io/projected/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-kube-api-access-hjjkv\") pod \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") "
Oct 30 22:01:06.127123 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.126977    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tmp-dir" pod "410f53f3-a093-4f1f-8cf6-fb7cd5eda739" (UID: "410f53f3-a093-4f1f-8cf6-fb7cd5eda739"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.127123 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127062    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-model-cache\") pod \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\" (UID: \"410f53f3-a093-4f1f-8cf6-fb7cd5eda739\") "
Oct 30 22:01:06.127766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127565    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-home" pod "6c05da20-a98a-4b76-9db6-44d92303fe12" (UID: "6c05da20-a98a-4b76-9db6-44d92303fe12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.127766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127737    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-model-cache" pod "410f53f3-a093-4f1f-8cf6-fb7cd5eda739" (UID: "410f53f3-a093-4f1f-8cf6-fb7cd5eda739"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.127971 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127763    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.127971 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127785    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-266wt\" (UniqueName: \"kubernetes.io/projected/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-kube-api-access-266wt\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.127971 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127799    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.127971 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127812    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.127971 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127826    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.127971 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127838    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.127971 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.127849    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.128731 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.128708    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-tmp-dir" pod "6c05da20-a98a-4b76-9db6-44d92303fe12" (UID: "6c05da20-a98a-4b76-9db6-44d92303fe12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.129316 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.129289    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c05da20-a98a-4b76-9db6-44d92303fe12-tls-certs" pod "6c05da20-a98a-4b76-9db6-44d92303fe12" (UID: "6c05da20-a98a-4b76-9db6-44d92303fe12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:01:06.129670 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.129647    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-dshm" pod "410f53f3-a093-4f1f-8cf6-fb7cd5eda739" (UID: "410f53f3-a093-4f1f-8cf6-fb7cd5eda739"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.129770 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.129731    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c05da20-a98a-4b76-9db6-44d92303fe12-kube-api-access-gh4x7" pod "6c05da20-a98a-4b76-9db6-44d92303fe12" (UID: "6c05da20-a98a-4b76-9db6-44d92303fe12"). InnerVolumeSpecName "kube-api-access-gh4x7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:06.129830 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.129817    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tls-certs" pod "410f53f3-a093-4f1f-8cf6-fb7cd5eda739" (UID: "410f53f3-a093-4f1f-8cf6-fb7cd5eda739"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:01:06.129959 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.129942    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-dshm" pod "6c05da20-a98a-4b76-9db6-44d92303fe12" (UID: "6c05da20-a98a-4b76-9db6-44d92303fe12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:06.131221 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.131202    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-kube-api-access-hjjkv" pod "410f53f3-a093-4f1f-8cf6-fb7cd5eda739" (UID: "410f53f3-a093-4f1f-8cf6-fb7cd5eda739"). InnerVolumeSpecName "kube-api-access-hjjkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:06.229239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229150    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c05da20-a98a-4b76-9db6-44d92303fe12-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.229239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229187    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.229239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229199    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.229239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229212    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh4x7\" (UniqueName: \"kubernetes.io/projected/6c05da20-a98a-4b76-9db6-44d92303fe12-kube-api-access-gh4x7\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.229239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229224    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.229606 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229243    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c05da20-a98a-4b76-9db6-44d92303fe12-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.229606 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229275    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjjkv\" (UniqueName: \"kubernetes.io/projected/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-kube-api-access-hjjkv\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.229606 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.229297    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/410f53f3-a093-4f1f-8cf6-fb7cd5eda739-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:06.350161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.350117    5817 generic.go:356] "Generic (PLEG): container finished" podID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerID="c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d" exitCode=0
Oct 30 22:01:06.350161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.350160    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"
Oct 30 22:01:06.350431 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.350199    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" event={"ID":"6c05da20-a98a-4b76-9db6-44d92303fe12","Type":"ContainerDied","Data":"c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d"}
Oct 30 22:01:06.350431 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.350270    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn" event={"ID":"6c05da20-a98a-4b76-9db6-44d92303fe12","Type":"ContainerDied","Data":"1a2416229dbf322dfdf154b37be8dda607e1840820eb2e4cc28be216f2733a4f"}
Oct 30 22:01:06.350431 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.350299    5817 scope.go:122] "RemoveContainer" containerID="c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d"
Oct 30 22:01:06.352974 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.352945    5817 generic.go:356] "Generic (PLEG): container finished" podID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerID="6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d" exitCode=0
Oct 30 22:01:06.354494 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.353094    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"
Oct 30 22:01:06.354494 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.353539    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" event={"ID":"410f53f3-a093-4f1f-8cf6-fb7cd5eda739","Type":"ContainerDied","Data":"6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d"}
Oct 30 22:01:06.354494 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.353626    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s" event={"ID":"410f53f3-a093-4f1f-8cf6-fb7cd5eda739","Type":"ContainerDied","Data":"a5b4afb5264921b82ece4f94016fc6cb0b13c33112c44f59418ce02c57f07276"}
Oct 30 22:01:06.357503 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.357476    5817 generic.go:356] "Generic (PLEG): container finished" podID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerID="800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928" exitCode=0
Oct 30 22:01:06.357665 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.357653    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" event={"ID":"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530","Type":"ContainerDied","Data":"800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928"}
Oct 30 22:01:06.357709 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.357677    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m" event={"ID":"d9c6d767-7ebe-46d5-8de8-fc46d3eaa530","Type":"ContainerDied","Data":"c54de483160b56b67c5b750e4c19d7995f747447fbcf498141287e17539f7979"}
Oct 30 22:01:06.357749 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.357720    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"
Oct 30 22:01:06.364826 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.364804    5817 scope.go:122] "RemoveContainer" containerID="c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d"
Oct 30 22:01:06.365160 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:06.365127    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d\": container with ID starting with c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d not found: ID does not exist" containerID="c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d"
Oct 30 22:01:06.365231 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.365166    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d"} err="failed to get container status \"c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d\": rpc error: code = NotFound desc = could not find container \"c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d\": container with ID starting with c74989c2148076a4970d0e150fa4b6cf8e4d758a1a8422be0fc6755a3e9c9b4d not found: ID does not exist"
Oct 30 22:01:06.365231 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.365193    5817 scope.go:122] "RemoveContainer" containerID="6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d"
Oct 30 22:01:06.381173 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.381149    5817 scope.go:122] "RemoveContainer" containerID="6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d"
Oct 30 22:01:06.381533 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:06.381514    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d\": container with ID starting with 6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d not found: ID does not exist" containerID="6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d"
Oct 30 22:01:06.381588 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.381543    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d"} err="failed to get container status \"6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d\": rpc error: code = NotFound desc = could not find container \"6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d\": container with ID starting with 6dfb373967ba306f5b68e6fc25a164221a47b81178868f3984411a03732a698d not found: ID does not exist"
Oct 30 22:01:06.381588 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.381559    5817 scope.go:122] "RemoveContainer" containerID="800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928"
Oct 30 22:01:06.391118 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.391099    5817 scope.go:122] "RemoveContainer" containerID="800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928"
Oct 30 22:01:06.391417 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:06.391396    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928\": container with ID starting with 800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928 not found: ID does not exist" containerID="800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928"
Oct 30 22:01:06.391500 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.391424    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928"} err="failed to get container status \"800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928\": rpc error: code = NotFound desc = could not find container \"800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928\": container with ID starting with 800b831bbb938c1595ae7ae0af23a0abec96bed9c057fe8cc87f5853a7cf4928 not found: ID does not exist"
Oct 30 22:01:06.567542 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.560074    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"]
Oct 30 22:01:06.568055 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.568027    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/simulator-w3-71aab5-kserve-57bcb8cdb7-mmhgn"]
Oct 30 22:01:06.638549 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.638512    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"]
Oct 30 22:01:06.653081 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.653048    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/distinct-w3-71aab5-kserve-69c7ddf58f-dgv8m"]
Oct 30 22:01:06.762432 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.762394    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"]
Oct 30 22:01:06.765444 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.765420    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-71aab5/premium-w3-71aab5-kserve-6d46bb4fd-5mv7s"]
Oct 30 22:01:06.847083 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.846990    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" path="/var/lib/kubelet/pods/410f53f3-a093-4f1f-8cf6-fb7cd5eda739/volumes"
Oct 30 22:01:06.847511 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.847485    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" path="/var/lib/kubelet/pods/6c05da20-a98a-4b76-9db6-44d92303fe12/volumes"
Oct 30 22:01:06.847925 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.847909    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" path="/var/lib/kubelet/pods/d9c6d767-7ebe-46d5-8de8-fc46d3eaa530/volumes"
Oct 30 22:01:06.857410 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:06.857380    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"]
Oct 30 22:01:08.914247 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:08.914210    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"]
Oct 30 22:01:08.914829 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:08.914517    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy" containerID="cri-o://b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77" gracePeriod=30
Oct 30 22:01:09.477299 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:09.477256    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" podUID="5b8250db-6367-4c18-8898-c0561e4364e1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:09.477480 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:09.477266    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" podUID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:10.260429 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.260390    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"]
Oct 30 22:01:10.261284 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261268    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy"
Oct 30 22:01:10.261331 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261287    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy"
Oct 30 22:01:10.261331 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261303    5817 cpu_manager.go:414] "containerMap: removing container" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main"
Oct 30 22:01:10.261331 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261311    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main"
Oct 30 22:01:10.261331 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261328    5817 cpu_manager.go:414] "containerMap: removing container" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261336    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261391    5817 cpu_manager.go:414] "containerMap: removing container" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261399    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261422    5817 cpu_manager.go:414] "containerMap: removing container" podUID="85da0cc8-f192-4bfa-8a98-1a5740da53c1" containerName="maas-api"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261429    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85da0cc8-f192-4bfa-8a98-1a5740da53c1" containerName="maas-api"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261453    5817 cpu_manager.go:414] "containerMap: removing container" podUID="dc6de55c-1998-478b-b65c-9b59c3b30a13" containerName="test-curl-11379-4667b9"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261460    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc6de55c-1998-478b-b65c-9b59c3b30a13" containerName="test-curl-11379-4667b9"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261487    5817 cpu_manager.go:414] "containerMap: removing container" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main"
Oct 30 22:01:10.261498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261494    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261625    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="85da0cc8-f192-4bfa-8a98-1a5740da53c1" containerName="maas-api"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261640    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc6de55c-1998-478b-b65c-9b59c3b30a13" containerName="test-curl-11379-4667b9"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261653    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261665    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9c6d767-7ebe-46d5-8de8-fc46d3eaa530" containerName="main"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261676    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261687    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261697    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c05da20-a98a-4b76-9db6-44d92303fe12" containerName="main"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261705    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy"
Oct 30 22:01:10.261805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.261720    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="410f53f3-a093-4f1f-8cf6-fb7cd5eda739" containerName="main"
Oct 30 22:01:10.268826 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.268804    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.319723 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.319666    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-28c192-serving-cert\""
Oct 30 22:01:10.323661 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.323628    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"]
Oct 30 22:01:10.395245 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.395210    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.395479 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.395342    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.395572 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.395545    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crzsd\" (UniqueName: \"kubernetes.io/projected/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-kube-api-access-crzsd\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.395636 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.395597    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-metrics-certs\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.499281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.498768    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.499281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.498930    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.499281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.499117    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crzsd\" (UniqueName: \"kubernetes.io/projected/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-kube-api-access-crzsd\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.499281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.499154    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-metrics-certs\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.506294 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.506256    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.506542 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:10.506414    5817 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-serving-cert: secret "maas-api-e2e-api-a-28c192-serving-cert" not found
Oct 30 22:01:10.506542 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:10.506478    5817 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls podName:c8e358dd-6be9-4c08-a230-f4c9a377bb3a nodeName:}" failed. No retries permitted until 2026-10-30 22:01:11.006457226 +0000 UTC m=+2806.294179751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls") pod "maas-api-e2e-api-a-28c192-74497557c6-bp89k" (UID: "c8e358dd-6be9-4c08-a230-f4c9a377bb3a") : secret "maas-api-e2e-api-a-28c192-serving-cert" not found
Oct 30 22:01:10.511883 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.511809    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-metrics-certs\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:10.566890 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:10.566854    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crzsd\" (UniqueName: \"kubernetes.io/projected/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-kube-api-access-crzsd\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:11.007917 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:11.007874    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:11.010318 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:11.010290    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls\") pod \"maas-api-e2e-api-a-28c192-74497557c6-bp89k\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:11.184601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:11.184558    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:11.558069 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:11.558029    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"]
Oct 30 22:01:11.560101 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:01:11.560072    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e358dd_6be9_4c08_a230_f4c9a377bb3a.slice/crio-b4d14fcecbf8e17583a4103e2a4229d15dd0015f118d38e0f0778459db745163 WatchSource:0}: Error finding container b4d14fcecbf8e17583a4103e2a4229d15dd0015f118d38e0f0778459db745163: Status 404 returned error can't find the container with id b4d14fcecbf8e17583a4103e2a4229d15dd0015f118d38e0f0778459db745163
Oct 30 22:01:12.405835 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:12.405764    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k" event={"ID":"c8e358dd-6be9-4c08-a230-f4c9a377bb3a","Type":"ContainerStarted","Data":"b4d14fcecbf8e17583a4103e2a4229d15dd0015f118d38e0f0778459db745163"}
Oct 30 22:01:13.415887 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:13.415765    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k" event={"ID":"c8e358dd-6be9-4c08-a230-f4c9a377bb3a","Type":"ContainerStarted","Data":"f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21"}
Oct 30 22:01:13.417211 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:13.417180    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:13.473864 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:13.473830    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"]
Oct 30 22:01:13.588615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:13.588558    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k" podStartSLOduration=2.710884298 podStartE2EDuration="3.588541244s" totalImagesPullingTime="877.656946ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:13.585719899 +0000 UTC m=+2808.873442444" watchObservedRunningTime="2026-10-30 22:01:13.588541244 +0000 UTC m=+2808.876263787"
Oct 30 22:01:13.592691 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:13.592662    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"]
Oct 30 22:01:14.204007 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.203981    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 22:01:14.271318 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271280    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-data\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271337    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9faf45e5-f84c-483c-8540-01dc105b932b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9faf45e5-f84c-483c-8540-01dc105b932b-istiod-ca-cert\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271451    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-certs\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271535    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-credential-socket\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271731 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271570    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-istio-token\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-istio-token\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271731 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271599    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-envoy\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271731 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271649    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-kube-api-access-6xdq2\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-kube-api-access-6xdq2\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271731 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271671    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-data" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:14.271940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271785    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-socket\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.271940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271832    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-certs" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:14.271940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271853    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-credential-socket" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:14.271940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271896    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9faf45e5-f84c-483c-8540-01dc105b932b-istiod-ca-cert" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:14.271940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.271923    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9faf45e5-f84c-483c-8540-01dc105b932b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9faf45e5-f84c-483c-8540-01dc105b932b-istio-podinfo\") pod \"9faf45e5-f84c-483c-8540-01dc105b932b\" (UID: \"9faf45e5-f84c-483c-8540-01dc105b932b\") "
Oct 30 22:01:14.272176 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.272098    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-socket" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:14.272497 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.272479    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.272568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.272499    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9faf45e5-f84c-483c-8540-01dc105b932b-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.272568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.272510    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.272568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.272525    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.272568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.272540    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.274281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.274097    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9faf45e5-f84c-483c-8540-01dc105b932b-istio-podinfo" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:01:14.274593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.274574    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-envoy" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:14.274593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.274584    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-istio-token" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:14.274715 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.274701    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-kube-api-access-6xdq2" pod "9faf45e5-f84c-483c-8540-01dc105b932b" (UID: "9faf45e5-f84c-483c-8540-01dc105b932b"). InnerVolumeSpecName "kube-api-access-6xdq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:14.373815 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.373780    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.373815 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.373812    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9faf45e5-f84c-483c-8540-01dc105b932b-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.374019 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.373825    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xdq2\" (UniqueName: \"kubernetes.io/projected/9faf45e5-f84c-483c-8540-01dc105b932b-kube-api-access-6xdq2\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.374019 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.373836    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9faf45e5-f84c-483c-8540-01dc105b932b-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:14.423755 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.423721    5817 generic.go:356] "Generic (PLEG): container finished" podID="9faf45e5-f84c-483c-8540-01dc105b932b" containerID="b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77" exitCode=0
Oct 30 22:01:14.424181 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.423772    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"
Oct 30 22:01:14.424181 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.423808    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" event={"ID":"9faf45e5-f84c-483c-8540-01dc105b932b","Type":"ContainerDied","Data":"b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77"}
Oct 30 22:01:14.424181 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.423866    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp" event={"ID":"9faf45e5-f84c-483c-8540-01dc105b932b","Type":"ContainerDied","Data":"17b63f4db72795d8e3dd2db72a56e269f1a7304829577255ab5f3152ffbd9665"}
Oct 30 22:01:14.424181 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.423894    5817 scope.go:122] "RemoveContainer" containerID="b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77"
Oct 30 22:01:14.437395 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.437375    5817 scope.go:122] "RemoveContainer" containerID="206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74"
Oct 30 22:01:14.446770 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.446747    5817 scope.go:122] "RemoveContainer" containerID="b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77"
Oct 30 22:01:14.447133 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:14.447115    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77\": container with ID starting with b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77 not found: ID does not exist" containerID="b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77"
Oct 30 22:01:14.447205 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.447142    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77"} err="failed to get container status \"b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77\": rpc error: code = NotFound desc = could not find container \"b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77\": container with ID starting with b87ea47fad172e0b10318bc215c6cedc1a9112ce91d2e2536de8570ea312cf77 not found: ID does not exist"
Oct 30 22:01:14.447205 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.447162    5817 scope.go:122] "RemoveContainer" containerID="206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74"
Oct 30 22:01:14.447471 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:14.447450    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74\": container with ID starting with 206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74 not found: ID does not exist" containerID="206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74"
Oct 30 22:01:14.447544 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.447477    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74"} err="failed to get container status \"206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74\": rpc error: code = NotFound desc = could not find container \"206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74\": container with ID starting with 206275b8095909b1903b37012b89625aea2a6916dbf0b3afebe0fa6700055f74 not found: ID does not exist"
Oct 30 22:01:14.588993 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.588901    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"]
Oct 30 22:01:14.596096 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.596064    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-40bb78-openshift-default-6b7769b5ff-p7nbp"]
Oct 30 22:01:14.848339 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:14.848253    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" path="/var/lib/kubelet/pods/9faf45e5-f84c-483c-8540-01dc105b932b/volumes"
Oct 30 22:01:16.338302 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:16.338264    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:01:18.474909 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:18.474869    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" podUID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" containerName="payload-processing" containerID="cri-o://333981d416721e2aadaacedea05ae5a810c6a3a7c41734c9de8d87b6d3ea4035" gracePeriod=25
Oct 30 22:01:18.593674 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:18.593635    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" podUID="c67beb8a-dc3d-47e0-a876-6d6197f22003" containerName="payload-pre-processing" containerID="cri-o://0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262" gracePeriod=25
Oct 30 22:01:19.477786 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:19.477745    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" podUID="5b8250db-6367-4c18-8898-c0561e4364e1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:19.478213 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:19.477814    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" podUID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:19.574550 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:19.574508    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" podUID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:19.574719 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:19.574514    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" podUID="c67beb8a-dc3d-47e0-a876-6d6197f22003" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:20.437625 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:20.437592    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:01:20.883707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:20.883673    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"]
Oct 30 22:01:21.002402 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:21.002365    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"]
Oct 30 22:01:22.387859 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.387820    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main" containerID="cri-o://75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023" gracePeriod=45
Oct 30 22:01:22.793399 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.793365    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 22:01:22.896883 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.896827    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-tmp-dir\") pod \"57f3244d-34b1-4025-8777-5db36836e0c8\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") "
Oct 30 22:01:22.896883 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.896876    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-dshm\") pod \"57f3244d-34b1-4025-8777-5db36836e0c8\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") "
Oct 30 22:01:22.897104 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.896943    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-model-cache\") pod \"57f3244d-34b1-4025-8777-5db36836e0c8\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") "
Oct 30 22:01:22.897104 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.897042    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57f3244d-34b1-4025-8777-5db36836e0c8-kube-api-access-fnf97\" (UniqueName: \"kubernetes.io/projected/57f3244d-34b1-4025-8777-5db36836e0c8-kube-api-access-fnf97\") pod \"57f3244d-34b1-4025-8777-5db36836e0c8\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") "
Oct 30 22:01:22.897226 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.897158    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57f3244d-34b1-4025-8777-5db36836e0c8-tls-certs\" (UniqueName: \"kubernetes.io/secret/57f3244d-34b1-4025-8777-5db36836e0c8-tls-certs\") pod \"57f3244d-34b1-4025-8777-5db36836e0c8\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") "
Oct 30 22:01:22.897226 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.897209    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-home\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-home\") pod \"57f3244d-34b1-4025-8777-5db36836e0c8\" (UID: \"57f3244d-34b1-4025-8777-5db36836e0c8\") "
Oct 30 22:01:22.897787 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.897762    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-home" pod "57f3244d-34b1-4025-8777-5db36836e0c8" (UID: "57f3244d-34b1-4025-8777-5db36836e0c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:22.898283 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.898056    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-tmp-dir" pod "57f3244d-34b1-4025-8777-5db36836e0c8" (UID: "57f3244d-34b1-4025-8777-5db36836e0c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:22.898454 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.898433    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-model-cache" pod "57f3244d-34b1-4025-8777-5db36836e0c8" (UID: "57f3244d-34b1-4025-8777-5db36836e0c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:22.901750 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.901721    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-dshm" pod "57f3244d-34b1-4025-8777-5db36836e0c8" (UID: "57f3244d-34b1-4025-8777-5db36836e0c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:01:22.902317 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.902291    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f3244d-34b1-4025-8777-5db36836e0c8-kube-api-access-fnf97" pod "57f3244d-34b1-4025-8777-5db36836e0c8" (UID: "57f3244d-34b1-4025-8777-5db36836e0c8"). InnerVolumeSpecName "kube-api-access-fnf97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:22.903266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.903235    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f3244d-34b1-4025-8777-5db36836e0c8-tls-certs" pod "57f3244d-34b1-4025-8777-5db36836e0c8" (UID: "57f3244d-34b1-4025-8777-5db36836e0c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:01:22.982984 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.982955    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 22:01:22.998837 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.998808    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57f3244d-34b1-4025-8777-5db36836e0c8-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:22.998837 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.998836    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:22.998837 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.998845    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:22.999057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.998853    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:22.999057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.998861    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57f3244d-34b1-4025-8777-5db36836e0c8-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:22.999057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:22.998872    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnf97\" (UniqueName: \"kubernetes.io/projected/57f3244d-34b1-4025-8777-5db36836e0c8-kube-api-access-fnf97\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:23.080263 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.080235    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.099685    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b8250db-6367-4c18-8898-c0561e4364e1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8250db-6367-4c18-8898-c0561e4364e1-plugins-config-volume\") pod \"5b8250db-6367-4c18-8898-c0561e4364e1\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") "
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.099810    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b8250db-6367-4c18-8898-c0561e4364e1-kube-api-access-jpz92\" (UniqueName: \"kubernetes.io/projected/5b8250db-6367-4c18-8898-c0561e4364e1-kube-api-access-jpz92\") pod \"5b8250db-6367-4c18-8898-c0561e4364e1\" (UID: \"5b8250db-6367-4c18-8898-c0561e4364e1\") "
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.100009    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae986a90-e7b8-4b48-a0ae-aa39e6524693-kube-api-access-dsg62\" (UniqueName: \"kubernetes.io/projected/ae986a90-e7b8-4b48-a0ae-aa39e6524693-kube-api-access-dsg62\") pod \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") "
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.100070    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae986a90-e7b8-4b48-a0ae-aa39e6524693-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae986a90-e7b8-4b48-a0ae-aa39e6524693-plugins-config-volume\") pod \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\" (UID: \"ae986a90-e7b8-4b48-a0ae-aa39e6524693\") "
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.100074    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b8250db-6367-4c18-8898-c0561e4364e1-plugins-config-volume" pod "5b8250db-6367-4c18-8898-c0561e4364e1" (UID: "5b8250db-6367-4c18-8898-c0561e4364e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.100571    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae986a90-e7b8-4b48-a0ae-aa39e6524693-plugins-config-volume" pod "ae986a90-e7b8-4b48-a0ae-aa39e6524693" (UID: "ae986a90-e7b8-4b48-a0ae-aa39e6524693"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.100718    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b8250db-6367-4c18-8898-c0561e4364e1-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:23.100781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.100743    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae986a90-e7b8-4b48-a0ae-aa39e6524693-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:23.102498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.102472    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae986a90-e7b8-4b48-a0ae-aa39e6524693-kube-api-access-dsg62" pod "ae986a90-e7b8-4b48-a0ae-aa39e6524693" (UID: "ae986a90-e7b8-4b48-a0ae-aa39e6524693"). InnerVolumeSpecName "kube-api-access-dsg62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:23.102641 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.102620    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b8250db-6367-4c18-8898-c0561e4364e1-kube-api-access-jpz92" pod "5b8250db-6367-4c18-8898-c0561e4364e1" (UID: "5b8250db-6367-4c18-8898-c0561e4364e1"). InnerVolumeSpecName "kube-api-access-jpz92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:23.202270 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.202236    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsg62\" (UniqueName: \"kubernetes.io/projected/ae986a90-e7b8-4b48-a0ae-aa39e6524693-kube-api-access-dsg62\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:23.202270 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.202266    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpz92\" (UniqueName: \"kubernetes.io/projected/5b8250db-6367-4c18-8898-c0561e4364e1-kube-api-access-jpz92\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:23.505075 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.505042    5817 generic.go:356] "Generic (PLEG): container finished" podID="57f3244d-34b1-4025-8777-5db36836e0c8" containerID="75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023" exitCode=0
Oct 30 22:01:23.505548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.505095    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"
Oct 30 22:01:23.505548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.505138    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" event={"ID":"57f3244d-34b1-4025-8777-5db36836e0c8","Type":"ContainerDied","Data":"75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023"}
Oct 30 22:01:23.505548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.505183    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh" event={"ID":"57f3244d-34b1-4025-8777-5db36836e0c8","Type":"ContainerDied","Data":"7a025815b492823e488194178b5f4d80c94e3e9d9afd3119edaccb723d96b890"}
Oct 30 22:01:23.505548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.505204    5817 scope.go:122] "RemoveContainer" containerID="75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023"
Oct 30 22:01:23.507087 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.507065    5817 generic.go:356] "Generic (PLEG): container finished" podID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" containerID="49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684" exitCode=0
Oct 30 22:01:23.507216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.507135    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"
Oct 30 22:01:23.507293 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.507256    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" event={"ID":"ae986a90-e7b8-4b48-a0ae-aa39e6524693","Type":"ContainerDied","Data":"49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684"}
Oct 30 22:01:23.507368 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.507294    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj" event={"ID":"ae986a90-e7b8-4b48-a0ae-aa39e6524693","Type":"ContainerDied","Data":"6e81e42bcf1503f5abd741ecaa64e7d2f86da6260d5b4a33f521631f3c121d74"}
Oct 30 22:01:23.508929 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.508903    5817 generic.go:356] "Generic (PLEG): container finished" podID="5b8250db-6367-4c18-8898-c0561e4364e1" containerID="e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef" exitCode=0
Oct 30 22:01:23.509043 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.508938    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" event={"ID":"5b8250db-6367-4c18-8898-c0561e4364e1","Type":"ContainerDied","Data":"e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef"}
Oct 30 22:01:23.509043 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.508988    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr" event={"ID":"5b8250db-6367-4c18-8898-c0561e4364e1","Type":"ContainerDied","Data":"70fa451d4e964dab98060b66d321e276c4a03e70e8d1959587eb2caf43bf847e"}
Oct 30 22:01:23.509217 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.509193    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"
Oct 30 22:01:23.523422 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.523263    5817 scope.go:122] "RemoveContainer" containerID="75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023"
Oct 30 22:01:23.523714 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:23.523687    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023\": container with ID starting with 75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023 not found: ID does not exist" containerID="75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023"
Oct 30 22:01:23.523795 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.523726    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023"} err="failed to get container status \"75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023\": rpc error: code = NotFound desc = could not find container \"75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023\": container with ID starting with 75b811ad7b6d3665c36381810e00a58eea5c84d0ee6731b2d4fb29d3135f2023 not found: ID does not exist"
Oct 30 22:01:23.523795 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.523752    5817 scope.go:122] "RemoveContainer" containerID="49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684"
Oct 30 22:01:23.563474 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.563452    5817 scope.go:122] "RemoveContainer" containerID="49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684"
Oct 30 22:01:23.563828 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:23.563798    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684\": container with ID starting with 49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684 not found: ID does not exist" containerID="49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684"
Oct 30 22:01:23.563928 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.563835    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684"} err="failed to get container status \"49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684\": rpc error: code = NotFound desc = could not find container \"49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684\": container with ID starting with 49d7d45fc07cc94443482102514eeab6286106b81ba55b46b5136f2820d1a684 not found: ID does not exist"
Oct 30 22:01:23.563928 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.563858    5817 scope.go:122] "RemoveContainer" containerID="e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef"
Oct 30 22:01:23.575092 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.575070    5817 scope.go:122] "RemoveContainer" containerID="e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef"
Oct 30 22:01:23.575513 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:23.575478    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef\": container with ID starting with e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef not found: ID does not exist" containerID="e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef"
Oct 30 22:01:23.575579 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.575520    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef"} err="failed to get container status \"e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef\": rpc error: code = NotFound desc = could not find container \"e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef\": container with ID starting with e41b32966f6e3d61318b15c591a149e758e3a34a2b3ea251c0d8ec15f490fcef not found: ID does not exist"
Oct 30 22:01:23.674085 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.673934    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"]
Oct 30 22:01:23.678277 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.678243    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-71aab5-7b846bc76c-zsznr"]
Oct 30 22:01:23.784975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.784938    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"]
Oct 30 22:01:23.788645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.788613    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-19dd4f/premium-w1-19dd4f-kserve-7bdff55b77-hjjwh"]
Oct 30 22:01:23.912403 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.912324    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"]
Oct 30 22:01:23.916672 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:23.916616    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-71aab5-66b687564b-fzsdj"]
Oct 30 22:01:24.872369 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:24.862608    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" path="/var/lib/kubelet/pods/57f3244d-34b1-4025-8777-5db36836e0c8/volumes"
Oct 30 22:01:24.872369 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:24.863310    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b8250db-6367-4c18-8898-c0561e4364e1" path="/var/lib/kubelet/pods/5b8250db-6367-4c18-8898-c0561e4364e1/volumes"
Oct 30 22:01:24.898844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:24.886687    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" path="/var/lib/kubelet/pods/ae986a90-e7b8-4b48-a0ae-aa39e6524693/volumes"
Oct 30 22:01:25.884881 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:25.884837    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7" podUID="00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" containerName="payload-processing" containerID="cri-o://fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140" gracePeriod=25
Oct 30 22:01:26.003636 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.003604    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc" podUID="9d7ba991-413e-4ade-a5e7-60ee2ef3446f" containerName="payload-pre-processing" containerID="cri-o://421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843" gracePeriod=25
Oct 30 22:01:26.157910 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.157847    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 22:01:26.252959 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.252933    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-plugins-config-volume\") pod \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") "
Oct 30 22:01:26.253074 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.252998    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-kube-api-access-87cqg\" (UniqueName: \"kubernetes.io/projected/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-kube-api-access-87cqg\") pod \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\" (UID: \"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd\") "
Oct 30 22:01:26.253380 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.253337    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-plugins-config-volume" pod "00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" (UID: "00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:26.253812 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.253774    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:26.255578 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.255547    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-kube-api-access-87cqg" pod "00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" (UID: "00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd"). InnerVolumeSpecName "kube-api-access-87cqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:26.265097 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.265077    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 22:01:26.354429 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.354393    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-plugins-config-volume\") pod \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") "
Oct 30 22:01:26.354634 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.354484    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-kube-api-access-8qr45\" (UniqueName: \"kubernetes.io/projected/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-kube-api-access-8qr45\") pod \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\" (UID: \"9d7ba991-413e-4ade-a5e7-60ee2ef3446f\") "
Oct 30 22:01:26.355045 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.355017    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-plugins-config-volume" pod "9d7ba991-413e-4ade-a5e7-60ee2ef3446f" (UID: "9d7ba991-413e-4ade-a5e7-60ee2ef3446f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:26.355157 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.355030    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87cqg\" (UniqueName: \"kubernetes.io/projected/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd-kube-api-access-87cqg\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:26.357026 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.357000    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-kube-api-access-8qr45" pod "9d7ba991-413e-4ade-a5e7-60ee2ef3446f" (UID: "9d7ba991-413e-4ade-a5e7-60ee2ef3446f"). InnerVolumeSpecName "kube-api-access-8qr45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:26.456497 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.456457    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qr45\" (UniqueName: \"kubernetes.io/projected/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-kube-api-access-8qr45\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:26.456497 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.456490    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d7ba991-413e-4ade-a5e7-60ee2ef3446f-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:26.541961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.541924    5817 generic.go:356] "Generic (PLEG): container finished" podID="9d7ba991-413e-4ade-a5e7-60ee2ef3446f" containerID="421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843" exitCode=0
Oct 30 22:01:26.542162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.541974    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"
Oct 30 22:01:26.542162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.542011    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc" event={"ID":"9d7ba991-413e-4ade-a5e7-60ee2ef3446f","Type":"ContainerDied","Data":"421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843"}
Oct 30 22:01:26.542162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.542083    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc" event={"ID":"9d7ba991-413e-4ade-a5e7-60ee2ef3446f","Type":"ContainerDied","Data":"61866d70556ba70d9d9a3067003d5e2f360d420813d42295cc6e83e89697f970"}
Oct 30 22:01:26.542162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.542112    5817 scope.go:122] "RemoveContainer" containerID="421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843"
Oct 30 22:01:26.543609 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.543587    5817 generic.go:356] "Generic (PLEG): container finished" podID="00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" containerID="fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140" exitCode=0
Oct 30 22:01:26.543728 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.543615    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7" event={"ID":"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd","Type":"ContainerDied","Data":"fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140"}
Oct 30 22:01:26.543728 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.543635    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"
Oct 30 22:01:26.543728 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.543669    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7" event={"ID":"00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd","Type":"ContainerDied","Data":"648aa1f7cb0a75547527bc95e38bce032c3243dcb9cbb183188a3c08bbc8d67c"}
Oct 30 22:01:26.555739 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.555720    5817 scope.go:122] "RemoveContainer" containerID="421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843"
Oct 30 22:01:26.556056 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:26.556035    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843\": container with ID starting with 421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843 not found: ID does not exist" containerID="421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843"
Oct 30 22:01:26.556128 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.556064    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843"} err="failed to get container status \"421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843\": rpc error: code = NotFound desc = could not find container \"421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843\": container with ID starting with 421a3b3ad927f3e69fc22d127dbfe65282f20963d17f405837db5c54a2da1843 not found: ID does not exist"
Oct 30 22:01:26.556128 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.556107    5817 scope.go:122] "RemoveContainer" containerID="fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140"
Oct 30 22:01:26.566655 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.566639    5817 scope.go:122] "RemoveContainer" containerID="fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140"
Oct 30 22:01:26.566916 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:26.566895    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140\": container with ID starting with fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140 not found: ID does not exist" containerID="fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140"
Oct 30 22:01:26.566967 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.566930    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140"} err="failed to get container status \"fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140\": rpc error: code = NotFound desc = could not find container \"fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140\": container with ID starting with fac65b99745b02e5e6a7acd89b90996eaf0604aba3eef0ae8a5d37a88c42d140 not found: ID does not exist"
Oct 30 22:01:26.701825 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.701785    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"]
Oct 30 22:01:26.710787 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.710715    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c4e874-55fcd87f7f-k7mvc"]
Oct 30 22:01:26.823244 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.823210    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"]
Oct 30 22:01:26.832201 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.832174    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c4e874-5d54cb86c8-t88d7"]
Oct 30 22:01:26.844061 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.844028    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" path="/var/lib/kubelet/pods/00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd/volumes"
Oct 30 22:01:26.844588 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:26.844565    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d7ba991-413e-4ade-a5e7-60ee2ef3446f" path="/var/lib/kubelet/pods/9d7ba991-413e-4ade-a5e7-60ee2ef3446f/volumes"
Oct 30 22:01:29.573625 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:29.573589    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" podUID="c67beb8a-dc3d-47e0-a876-6d6197f22003" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:29.574124 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:29.573595    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" podUID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:01:38.620323 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.620294    5817 generic.go:356] "Generic (PLEG): container finished" podID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" containerID="333981d416721e2aadaacedea05ae5a810c6a3a7c41734c9de8d87b6d3ea4035" exitCode=0
Oct 30 22:01:38.620674 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.620376    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" event={"ID":"7fce027b-4ba9-4fb6-a01f-debcc062df5c","Type":"ContainerDied","Data":"333981d416721e2aadaacedea05ae5a810c6a3a7c41734c9de8d87b6d3ea4035"}
Oct 30 22:01:38.756938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.756909    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 22:01:38.850139 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.850107    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7fce027b-4ba9-4fb6-a01f-debcc062df5c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fce027b-4ba9-4fb6-a01f-debcc062df5c-plugins-config-volume\") pod \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") "
Oct 30 22:01:38.850295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.850205    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fce027b-4ba9-4fb6-a01f-debcc062df5c-kube-api-access-np44t\" (UniqueName: \"kubernetes.io/projected/7fce027b-4ba9-4fb6-a01f-debcc062df5c-kube-api-access-np44t\") pod \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\" (UID: \"7fce027b-4ba9-4fb6-a01f-debcc062df5c\") "
Oct 30 22:01:38.850492 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.850468    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fce027b-4ba9-4fb6-a01f-debcc062df5c-plugins-config-volume" pod "7fce027b-4ba9-4fb6-a01f-debcc062df5c" (UID: "7fce027b-4ba9-4fb6-a01f-debcc062df5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:38.850637 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.850617    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fce027b-4ba9-4fb6-a01f-debcc062df5c-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:38.852268 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.852244    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fce027b-4ba9-4fb6-a01f-debcc062df5c-kube-api-access-np44t" pod "7fce027b-4ba9-4fb6-a01f-debcc062df5c" (UID: "7fce027b-4ba9-4fb6-a01f-debcc062df5c"). InnerVolumeSpecName "kube-api-access-np44t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:38.866816 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.866796    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 22:01:38.952259 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:38.952224    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np44t\" (UniqueName: \"kubernetes.io/projected/7fce027b-4ba9-4fb6-a01f-debcc062df5c-kube-api-access-np44t\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:39.053761 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.053681    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c67beb8a-dc3d-47e0-a876-6d6197f22003-kube-api-access-bxh7p\" (UniqueName: \"kubernetes.io/projected/c67beb8a-dc3d-47e0-a876-6d6197f22003-kube-api-access-bxh7p\") pod \"c67beb8a-dc3d-47e0-a876-6d6197f22003\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") "
Oct 30 22:01:39.053903 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.053788    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c67beb8a-dc3d-47e0-a876-6d6197f22003-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67beb8a-dc3d-47e0-a876-6d6197f22003-plugins-config-volume\") pod \"c67beb8a-dc3d-47e0-a876-6d6197f22003\" (UID: \"c67beb8a-dc3d-47e0-a876-6d6197f22003\") "
Oct 30 22:01:39.054168 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.054145    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67beb8a-dc3d-47e0-a876-6d6197f22003-plugins-config-volume" pod "c67beb8a-dc3d-47e0-a876-6d6197f22003" (UID: "c67beb8a-dc3d-47e0-a876-6d6197f22003"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:01:39.054327 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.054313    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c67beb8a-dc3d-47e0-a876-6d6197f22003-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:39.055895 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.055875    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c67beb8a-dc3d-47e0-a876-6d6197f22003-kube-api-access-bxh7p" pod "c67beb8a-dc3d-47e0-a876-6d6197f22003" (UID: "c67beb8a-dc3d-47e0-a876-6d6197f22003"). InnerVolumeSpecName "kube-api-access-bxh7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:01:39.155939 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.155904    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxh7p\" (UniqueName: \"kubernetes.io/projected/c67beb8a-dc3d-47e0-a876-6d6197f22003-kube-api-access-bxh7p\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:01:39.627663 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.627629    5817 generic.go:356] "Generic (PLEG): container finished" podID="c67beb8a-dc3d-47e0-a876-6d6197f22003" containerID="0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262" exitCode=0
Oct 30 22:01:39.628115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.627694    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"
Oct 30 22:01:39.628115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.627720    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" event={"ID":"c67beb8a-dc3d-47e0-a876-6d6197f22003","Type":"ContainerDied","Data":"0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262"}
Oct 30 22:01:39.628115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.627788    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f" event={"ID":"c67beb8a-dc3d-47e0-a876-6d6197f22003","Type":"ContainerDied","Data":"42cc1eec12272edda559afa3a4cdd3537dac90dce6db66a0a74ef1eafc736115"}
Oct 30 22:01:39.628115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.627821    5817 scope.go:122] "RemoveContainer" containerID="0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262"
Oct 30 22:01:39.629392 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.629341    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w" event={"ID":"7fce027b-4ba9-4fb6-a01f-debcc062df5c","Type":"ContainerDied","Data":"2644f52760cb047dd50666ce69704338600433e5ffb7ea41aa01553c83cd2f11"}
Oct 30 22:01:39.629497 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.629390    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"
Oct 30 22:01:39.639192 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.639175    5817 scope.go:122] "RemoveContainer" containerID="0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262"
Oct 30 22:01:39.639569 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:39.639541    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262\": container with ID starting with 0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262 not found: ID does not exist" containerID="0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262"
Oct 30 22:01:39.639660 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.639580    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262"} err="failed to get container status \"0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262\": rpc error: code = NotFound desc = could not find container \"0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262\": container with ID starting with 0e7465da3bc8cedfd1c0ebdf3c25e9bf743ed398a79afca16ef427bd454fc262 not found: ID does not exist"
Oct 30 22:01:39.639660 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.639602    5817 scope.go:122] "RemoveContainer" containerID="333981d416721e2aadaacedea05ae5a810c6a3a7c41734c9de8d87b6d3ea4035"
Oct 30 22:01:39.789408 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.789304    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"]
Oct 30 22:01:39.836261 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.836228    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-19dd4f-578b5c44c9-cm67f"]
Oct 30 22:01:39.899317 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.899280    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"]
Oct 30 22:01:39.902635 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:39.902610    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-19dd4f-799c9f8c6-4lp4w"]
Oct 30 22:01:40.844042 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:40.844007    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" path="/var/lib/kubelet/pods/7fce027b-4ba9-4fb6-a01f-debcc062df5c/volumes"
Oct 30 22:01:40.844463 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:40.844448    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c67beb8a-dc3d-47e0-a876-6d6197f22003" path="/var/lib/kubelet/pods/c67beb8a-dc3d-47e0-a876-6d6197f22003/volumes"
Oct 30 22:01:46.382375 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:01:46.382309    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:01:58.796132 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.796097    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"]
Oct 30 22:01:58.797616 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.797587    5817 cpu_manager.go:414] "containerMap: removing container" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy"
Oct 30 22:01:58.797780 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.797761    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy"
Oct 30 22:01:58.797947 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.797928    5817 cpu_manager.go:414] "containerMap: removing container" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy"
Oct 30 22:01:58.798053 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798041    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy"
Oct 30 22:01:58.798146 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798137    5817 cpu_manager.go:414] "containerMap: removing container" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main"
Oct 30 22:01:58.798226 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798217    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main"
Oct 30 22:01:58.798326 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798317    5817 cpu_manager.go:414] "containerMap: removing container" podUID="c67beb8a-dc3d-47e0-a876-6d6197f22003" containerName="payload-pre-processing"
Oct 30 22:01:58.798434 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798419    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c67beb8a-dc3d-47e0-a876-6d6197f22003" containerName="payload-pre-processing"
Oct 30 22:01:58.798512 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798500    5817 cpu_manager.go:414] "containerMap: removing container" podUID="00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" containerName="payload-processing"
Oct 30 22:01:58.798589 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798579    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" containerName="payload-processing"
Oct 30 22:01:58.798678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798668    5817 cpu_manager.go:414] "containerMap: removing container" podUID="5b8250db-6367-4c18-8898-c0561e4364e1" containerName="payload-pre-processing"
Oct 30 22:01:58.798750 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798739    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b8250db-6367-4c18-8898-c0561e4364e1" containerName="payload-pre-processing"
Oct 30 22:01:58.798904 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798894    5817 cpu_manager.go:414] "containerMap: removing container" podUID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" containerName="payload-processing"
Oct 30 22:01:58.798977 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.798966    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" containerName="payload-processing"
Oct 30 22:01:58.799060 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799048    5817 cpu_manager.go:414] "containerMap: removing container" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy"
Oct 30 22:01:58.799161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799127    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a0ddf30-bcf7-4449-8871-9245159c0b09" containerName="istio-proxy"
Oct 30 22:01:58.799161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799149    5817 cpu_manager.go:414] "containerMap: removing container" podUID="9d7ba991-413e-4ade-a5e7-60ee2ef3446f" containerName="payload-pre-processing"
Oct 30 22:01:58.799161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799157    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d7ba991-413e-4ade-a5e7-60ee2ef3446f" containerName="payload-pre-processing"
Oct 30 22:01:58.799161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799172    5817 cpu_manager.go:414] "containerMap: removing container" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799179    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7f80f7d-26be-43b8-b355-9c7e8a9924d7" containerName="istio-proxy"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799192    5817 cpu_manager.go:414] "containerMap: removing container" podUID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" containerName="payload-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799200    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" containerName="payload-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799439    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="c67beb8a-dc3d-47e0-a876-6d6197f22003" containerName="payload-pre-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799454    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="00ca11db-e5cd-4e70-b5d8-1a85e62eb0fd" containerName="payload-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799468    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b8250db-6367-4c18-8898-c0561e4364e1" containerName="payload-pre-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799490    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae986a90-e7b8-4b48-a0ae-aa39e6524693" containerName="payload-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799501    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d7ba991-413e-4ade-a5e7-60ee2ef3446f" containerName="payload-pre-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799511    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fce027b-4ba9-4fb6-a01f-debcc062df5c" containerName="payload-processing"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799522    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799534    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="9faf45e5-f84c-483c-8540-01dc105b932b" containerName="istio-proxy"
Oct 30 22:01:58.799551 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.799544    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="57f3244d-34b1-4025-8777-5db36836e0c8" containerName="main"
Oct 30 22:01:58.803529 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.803502    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.854725 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.854703    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-87eb6a-openshift-default-dockercfg-h4p7v\""
Oct 30 22:01:58.903786 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.903752    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"]
Oct 30 22:01:58.932295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932263    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-credential-socket\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932478 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932308    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-certs\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932478 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932424    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-envoy\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932557 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932532    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjhv\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-kube-api-access-dsjhv\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932632 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932614    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-istio-token\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932749 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932737    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8972290a-50d8-4ad3-8998-88a670362dae-istiod-ca-cert\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932786 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932760    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-data\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932868 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932856    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-socket\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:58.932904 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:58.932894    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8972290a-50d8-4ad3-8998-88a670362dae-istio-podinfo\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034153    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8972290a-50d8-4ad3-8998-88a670362dae-istiod-ca-cert\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034388 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034195    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-data\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034388 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034283    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-socket\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034388 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034333    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8972290a-50d8-4ad3-8998-88a670362dae-istio-podinfo\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034560 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034439    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-credential-socket\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034560 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034487    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-certs\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034560 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034514    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-envoy\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034721 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034558    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjhv\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-kube-api-access-dsjhv\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034721 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034607    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-istio-token\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034721 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034705    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-data\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034771    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-socket\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034803    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-credential-socket\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.034911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.034833    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-certs\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.035276 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.035257    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8972290a-50d8-4ad3-8998-88a670362dae-istiod-ca-cert\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.036723 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.036704    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-envoy\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.036846 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.036829    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8972290a-50d8-4ad3-8998-88a670362dae-istio-podinfo\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.100628 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.100567    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjhv\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-kube-api-access-dsjhv\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.101190 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.101167    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-istio-token\") pod \"e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") " pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.121225 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.121204    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:01:59.561378 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.561324    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"]
Oct 30 22:01:59.588151 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:01:59.588121    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8972290a_50d8_4ad3_8998_88a670362dae.slice/crio-5d4e9023ad16a9b18df4e38d1b34456e651d486e7845780098fad052a4b2e854 WatchSource:0}: Error finding container 5d4e9023ad16a9b18df4e38d1b34456e651d486e7845780098fad052a4b2e854: Status 404 returned error can't find the container with id 5d4e9023ad16a9b18df4e38d1b34456e651d486e7845780098fad052a4b2e854
Oct 30 22:01:59.590634 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.590601    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:59.590706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.590671    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:59.590706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.590698    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:01:59.802024 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.801987    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" event={"ID":"8972290a-50d8-4ad3-8998-88a670362dae","Type":"ContainerStarted","Data":"18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a"}
Oct 30 22:01:59.802453 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.802040    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" event={"ID":"8972290a-50d8-4ad3-8998-88a670362dae","Type":"ContainerStarted","Data":"5d4e9023ad16a9b18df4e38d1b34456e651d486e7845780098fad052a4b2e854"}
Oct 30 22:01:59.923698 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:01:59.923549    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podStartSLOduration=1.923532143 podStartE2EDuration="1.923532143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:59.921315279 +0000 UTC m=+2855.209037825" watchObservedRunningTime="2026-10-30 22:01:59.923532143 +0000 UTC m=+2855.211254689"
Oct 30 22:02:00.122165 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:00.122137    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:02:00.127761 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:00.127738    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:02:00.808279 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:00.808243    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:02:00.809485 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:00.809460    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:02:00.815153 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:00.815129    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"]
Oct 30 22:02:08.662138 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:08.662101    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"]
Oct 30 22:02:08.662630 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:08.662407    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy" containerID="cri-o://47bdaa4c3b0260e7397cd814c97890c5089b9bdb51ba008584d8545bf5899bd9" gracePeriod=30
Oct 30 22:02:13.909785 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:13.909743    5817 generic.go:356] "Generic (PLEG): container finished" podID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerID="47bdaa4c3b0260e7397cd814c97890c5089b9bdb51ba008584d8545bf5899bd9" exitCode=0
Oct 30 22:02:13.910215 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:13.909815    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" event={"ID":"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d","Type":"ContainerDied","Data":"47bdaa4c3b0260e7397cd814c97890c5089b9bdb51ba008584d8545bf5899bd9"}
Oct 30 22:02:13.910215 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:13.909899    5817 scope.go:122] "RemoveContainer" containerID="86a49f91ea060a774a03b9d6257eb1b9f13aec532207af6ecb13acdcc7bdeabd"
Oct 30 22:02:13.960882 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:13.960856    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 22:02:14.130081 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130046    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-socket\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.130252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130162    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istiod-ca-cert\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.130315 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130291    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-data\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.130379 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130326    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-certs\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.130445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130340    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-socket" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:14.130445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130394    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-kube-api-access-hx5nj\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-kube-api-access-hx5nj\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.130557 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130452    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-envoy\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.130557 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130479    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-token\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-token\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.130557 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130524    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-podinfo\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130556    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-data" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130593    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-certs" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130619    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istiod-ca-cert" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.130645    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-credential-socket\") pod \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\" (UID: \"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d\") "
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.131133    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-credential-socket" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.131313    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.131338    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.131394    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.131409    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.131767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.131423    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.133092 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.133063    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-podinfo" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:14.133202 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.133185    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-kube-api-access-hx5nj" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "kube-api-access-hx5nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:14.133319 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.133287    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-envoy" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:14.133481 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.133329    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-token" pod "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" (UID: "2b8839ba-a89c-43b8-a9eb-c4c96b232b2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:14.179489 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.179404    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"]
Oct 30 22:02:14.180321 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.180045    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"]
Oct 30 22:02:14.183422 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.183395    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"]
Oct 30 22:02:14.232831 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.232797    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.232831 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.232822    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.232831 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.232832    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.233041 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.232842    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx5nj\" (UniqueName: \"kubernetes.io/projected/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d-kube-api-access-hx5nj\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:14.917547 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.917523    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"
Oct 30 22:02:14.917932 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.917542    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb" event={"ID":"2b8839ba-a89c-43b8-a9eb-c4c96b232b2d","Type":"ContainerDied","Data":"975bb501e41c14144363ec5b11a1c9134234dc32b306623b97a60821f84c1f5b"}
Oct 30 22:02:14.917932 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:14.917599    5817 scope.go:122] "RemoveContainer" containerID="47bdaa4c3b0260e7397cd814c97890c5089b9bdb51ba008584d8545bf5899bd9"
Oct 30 22:02:15.091151 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.091108    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"]
Oct 30 22:02:15.137459 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.137380    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-ca0a55-openshift-default-b786f55bf-v8xxb"]
Oct 30 22:02:15.477244 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.477213    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"]
Oct 30 22:02:15.477496 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.477455    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4" podUID="122cf522-ccbf-409b-9956-baf8a6ff103a" containerName="maas-api" containerID="cri-o://c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604" gracePeriod=30
Oct 30 22:02:15.749093 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.749060    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 22:02:15.835452 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.835404    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" containerID="cri-o://eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857" gracePeriod=45
Oct 30 22:02:15.853522 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.853496    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-metrics-certs\") pod \"122cf522-ccbf-409b-9956-baf8a6ff103a\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") "
Oct 30 22:02:15.853620 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.853541    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls\") pod \"122cf522-ccbf-409b-9956-baf8a6ff103a\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") "
Oct 30 22:02:15.853620 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.853575    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/122cf522-ccbf-409b-9956-baf8a6ff103a-kube-api-access-skw7x\" (UniqueName: \"kubernetes.io/projected/122cf522-ccbf-409b-9956-baf8a6ff103a-kube-api-access-skw7x\") pod \"122cf522-ccbf-409b-9956-baf8a6ff103a\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") "
Oct 30 22:02:15.853777 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.853758    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/122cf522-ccbf-409b-9956-baf8a6ff103a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/122cf522-ccbf-409b-9956-baf8a6ff103a-openshift-service-ca\") pod \"122cf522-ccbf-409b-9956-baf8a6ff103a\" (UID: \"122cf522-ccbf-409b-9956-baf8a6ff103a\") "
Oct 30 22:02:15.854235 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.854168    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122cf522-ccbf-409b-9956-baf8a6ff103a-openshift-service-ca" pod "122cf522-ccbf-409b-9956-baf8a6ff103a" (UID: "122cf522-ccbf-409b-9956-baf8a6ff103a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:15.854428 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.854405    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/122cf522-ccbf-409b-9956-baf8a6ff103a-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:15.855636 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.855614    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-metrics-certs" pod "122cf522-ccbf-409b-9956-baf8a6ff103a" (UID: "122cf522-ccbf-409b-9956-baf8a6ff103a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:15.855775 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.855760    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls" pod "122cf522-ccbf-409b-9956-baf8a6ff103a" (UID: "122cf522-ccbf-409b-9956-baf8a6ff103a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:15.855827 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.855807    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122cf522-ccbf-409b-9956-baf8a6ff103a-kube-api-access-skw7x" pod "122cf522-ccbf-409b-9956-baf8a6ff103a" (UID: "122cf522-ccbf-409b-9956-baf8a6ff103a"). InnerVolumeSpecName "kube-api-access-skw7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:15.933979 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.933947    5817 generic.go:356] "Generic (PLEG): container finished" podID="122cf522-ccbf-409b-9956-baf8a6ff103a" containerID="c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604" exitCode=0
Oct 30 22:02:15.934489 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.934003    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"
Oct 30 22:02:15.934489 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.934034    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4" event={"ID":"122cf522-ccbf-409b-9956-baf8a6ff103a","Type":"ContainerDied","Data":"c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604"}
Oct 30 22:02:15.934489 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.934091    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4" event={"ID":"122cf522-ccbf-409b-9956-baf8a6ff103a","Type":"ContainerDied","Data":"9a24ca64d03456e5af337d002c6f485ac0b38f60fdc8a3dd48cc0b2f262a58a8"}
Oct 30 22:02:15.934489 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.934122    5817 scope.go:122] "RemoveContainer" containerID="c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604"
Oct 30 22:02:15.949750 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.949725    5817 scope.go:122] "RemoveContainer" containerID="c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604"
Oct 30 22:02:15.950020 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.949990    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": dial tcp 10.134.0.64:8000: connect: connection refused"
Oct 30 22:02:15.950119 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:15.950065    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604\": container with ID starting with c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604 not found: ID does not exist" containerID="c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604"
Oct 30 22:02:15.950119 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.950088    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604"} err="failed to get container status \"c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604\": rpc error: code = NotFound desc = could not find container \"c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604\": container with ID starting with c24b6e154493757aac44235257d13a9bb1677f4a1cd9018bb8217290f5974604 not found: ID does not exist"
Oct 30 22:02:15.956458 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.956440    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:15.956583 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.956461    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/122cf522-ccbf-409b-9956-baf8a6ff103a-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:15.956583 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:15.956477    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skw7x\" (UniqueName: \"kubernetes.io/projected/122cf522-ccbf-409b-9956-baf8a6ff103a-kube-api-access-skw7x\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:16.102212 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.102178    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"]
Oct 30 22:02:16.108100 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.108075    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-19dd4f-f4f985d8d-zc2j4"]
Oct 30 22:02:16.113705 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.113683    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 22:02:16.158884 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.158861    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-dshm\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-dshm\") pod \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") "
Oct 30 22:02:16.158986 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.158897    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tmp-dir\") pod \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") "
Oct 30 22:02:16.158986 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.158959    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tls-certs\" (UniqueName: \"kubernetes.io/secret/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tls-certs\") pod \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") "
Oct 30 22:02:16.159076 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159062    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-model-cache\") pod \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") "
Oct 30 22:02:16.159172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159155    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-home\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-home\") pod \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") "
Oct 30 22:02:16.159234 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159183    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tmp-dir" pod "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" (UID: "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:16.159234 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159225    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-kube-api-access-bsjft\" (UniqueName: \"kubernetes.io/projected/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-kube-api-access-bsjft\") pod \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\" (UID: \"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa\") "
Oct 30 22:02:16.159408 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159384    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-model-cache" pod "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" (UID: "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:16.159520 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159481    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-home" pod "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" (UID: "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:16.159847 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159823    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:16.159940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159853    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:16.159940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.159868    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:16.161135 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.161115    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-dshm" pod "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" (UID: "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:16.161247 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.161171    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tls-certs" pod "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" (UID: "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:16.161311 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.161296    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-kube-api-access-bsjft" pod "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" (UID: "313a4a3a-5ea1-4090-b8d6-40f181ffd3aa"). InnerVolumeSpecName "kube-api-access-bsjft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:16.260957 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.260928    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:16.260957 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.260957    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:16.261145 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.260971    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsjft\" (UniqueName: \"kubernetes.io/projected/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa-kube-api-access-bsjft\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:16.338281 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:16.338201    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:02:16.847041 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.847010    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="122cf522-ccbf-409b-9956-baf8a6ff103a" path="/var/lib/kubelet/pods/122cf522-ccbf-409b-9956-baf8a6ff103a/volumes"
Oct 30 22:02:16.847628 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.847611    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" path="/var/lib/kubelet/pods/2b8839ba-a89c-43b8-a9eb-c4c96b232b2d/volumes"
Oct 30 22:02:16.942885 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.942858    5817 generic.go:356] "Generic (PLEG): container finished" podID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerID="eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857" exitCode=0
Oct 30 22:02:16.943277 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.942901    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"
Oct 30 22:02:16.943277 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.942946    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" event={"ID":"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa","Type":"ContainerDied","Data":"eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857"}
Oct 30 22:02:16.943277 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.942992    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt" event={"ID":"313a4a3a-5ea1-4090-b8d6-40f181ffd3aa","Type":"ContainerDied","Data":"407e9a209574d9abedfe87878f9388108be572d54d32b5239bfa9d08874224d1"}
Oct 30 22:02:16.943277 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.943014    5817 scope.go:122] "RemoveContainer" containerID="eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857"
Oct 30 22:02:16.952534 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.952513    5817 scope.go:122] "RemoveContainer" containerID="eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857"
Oct 30 22:02:16.952783 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:16.952765    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857\": container with ID starting with eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857 not found: ID does not exist" containerID="eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857"
Oct 30 22:02:16.952835 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:16.952789    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857"} err="failed to get container status \"eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857\": rpc error: code = NotFound desc = could not find container \"eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857\": container with ID starting with eebdfff3dd675b86cd89a666df1de5bcd4ce87d6f31f3b9404418e638c97d857 not found: ID does not exist"
Oct 30 22:02:17.108958 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:17.108880    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"]
Oct 30 22:02:17.155889 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:17.155859    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-a2dced/distinct-w6-a2dced-kserve-7dd455cdd-bwjkt"]
Oct 30 22:02:18.843807 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:18.843773    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" path="/var/lib/kubelet/pods/313a4a3a-5ea1-4090-b8d6-40f181ffd3aa/volumes"
Oct 30 22:02:19.839870 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.839836    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"]
Oct 30 22:02:19.840099 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.840073    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy" containerID="cri-o://28ca5ed569ca1ad44b9319d31535c33152d82e307fda7d6ecb66ab1cd4411de4" gracePeriod=30
Oct 30 22:02:19.843615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.843585    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"]
Oct 30 22:02:19.843838 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.843810    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy" containerID="cri-o://7769fe88a892751ec9bb095c4097637ee58968ae12b63d13a81dc0f6d756651e" gracePeriod=30
Oct 30 22:02:19.862240 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.862206    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"]
Oct 30 22:02:19.862492 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.862465    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy" containerID="cri-o://0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad" gracePeriod=30
Oct 30 22:02:19.886930 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.886896    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"]
Oct 30 22:02:19.887215 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:19.887186    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy" containerID="cri-o://6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1" gracePeriod=30
Oct 30 22:02:21.430916 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:21.430883    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"]
Oct 30 22:02:22.281383 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:22.281332    5817 patch_prober.go:29] interesting pod/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:02:22.281568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:22.281407    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:02:25.008555 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.008522    5817 generic.go:356] "Generic (PLEG): container finished" podID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerID="28ca5ed569ca1ad44b9319d31535c33152d82e307fda7d6ecb66ab1cd4411de4" exitCode=0
Oct 30 22:02:25.008999 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.008903    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" event={"ID":"a18e6ea2-6992-427d-8c0f-6244c76052d6","Type":"ContainerDied","Data":"28ca5ed569ca1ad44b9319d31535c33152d82e307fda7d6ecb66ab1cd4411de4"}
Oct 30 22:02:25.011432 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.011398    5817 generic.go:356] "Generic (PLEG): container finished" podID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerID="7769fe88a892751ec9bb095c4097637ee58968ae12b63d13a81dc0f6d756651e" exitCode=0
Oct 30 22:02:25.011558 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.011481    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" event={"ID":"fd7217ba-460e-45c6-804b-70ebd50c3d38","Type":"ContainerDied","Data":"7769fe88a892751ec9bb095c4097637ee58968ae12b63d13a81dc0f6d756651e"}
Oct 30 22:02:25.011558 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.011549    5817 scope.go:122] "RemoveContainer" containerID="5ca32526f9bb2988d78ae56c0e7cd218e8caf1cdc2545a58cc91e8e12b4c1d99"
Oct 30 22:02:25.013713 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.013693    5817 generic.go:356] "Generic (PLEG): container finished" podID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerID="0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad" exitCode=0
Oct 30 22:02:25.013793 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.013757    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" event={"ID":"9680477d-1d4e-45b5-b96b-f20a0044fd3e","Type":"ContainerDied","Data":"0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad"}
Oct 30 22:02:25.192672 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.192640    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.232786    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-certs\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.232881    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-data\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.232912    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-envoy\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.232949    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-socket\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.232994    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-kube-api-access-gsmbv\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-kube-api-access-gsmbv\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.233052    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-podinfo\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.233143    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-credential-socket\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.233203    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-token\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-token\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.233418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.233238    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istiod-ca-cert\") pod \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\" (UID: \"9680477d-1d4e-45b5-b96b-f20a0044fd3e\") "
Oct 30 22:02:25.234558 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.234113    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-socket" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.234558 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.234386    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-certs" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.234817 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.234776    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-data" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.236416 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.236377    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-credential-socket" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.236549 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.236484    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istiod-ca-cert" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:25.236949 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.236927    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.237047 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.236954    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.237047 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.236968    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.237047 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.236982    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.237047 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.236997    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.241189 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.241169    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-token" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.243383 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.243182    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-envoy" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.245563 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.245366    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-podinfo" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:25.245563 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.245487    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-kube-api-access-gsmbv" pod "9680477d-1d4e-45b5-b96b-f20a0044fd3e" (UID: "9680477d-1d4e-45b5-b96b-f20a0044fd3e"). InnerVolumeSpecName "kube-api-access-gsmbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.277025 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.276944    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:02:25.279790 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.279767    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 22:02:25.279884 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.279837    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"]
Oct 30 22:02:25.280046 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.280012    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy" containerID="cri-o://c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17" gracePeriod=30
Oct 30 22:02:25.283793 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.283771    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:02:25.338006 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.337975    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a18e6ea2-6992-427d-8c0f-6244c76052d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18e6ea2-6992-427d-8c0f-6244c76052d6-istiod-ca-cert\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.338166 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338051    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-socket\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.338166 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338141    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd7217ba-460e-45c6-804b-70ebd50c3d38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd7217ba-460e-45c6-804b-70ebd50c3d38-istiod-ca-cert\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.338390 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338209    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-token\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-token\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.338390 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338241    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-credential-socket\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.338390 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338285    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-envoy\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.338390 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338332    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-credential-socket\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.338390 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338333    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-socket" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338400    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18e6ea2-6992-427d-8c0f-6244c76052d6-istiod-ca-cert" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338421    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-envoy\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338474    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-data\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338500    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-data\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338538    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-kube-api-access-qdcw9\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-kube-api-access-qdcw9\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338540    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd7217ba-460e-45c6-804b-70ebd50c3d38-istiod-ca-cert" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338576    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-kube-api-access-7ws2m\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-kube-api-access-7ws2m\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338619    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-certs\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.338678 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338648    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-token\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-token\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338690    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-certs\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338728    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-podinfo\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338783    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-data\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338831    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-credential-socket\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338855    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-kube-api-access-k5j24\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-kube-api-access-k5j24\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338877    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-socket\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338922    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-socket\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.338976    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-envoy\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.339019    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-certs\") pod \"a18e6ea2-6992-427d-8c0f-6244c76052d6\" (UID: \"a18e6ea2-6992-427d-8c0f-6244c76052d6\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.339033    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-certs" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.339053    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-podinfo\") pod \"fd7217ba-460e-45c6-804b-70ebd50c3d38\" (UID: \"fd7217ba-460e-45c6-804b-70ebd50c3d38\") "
Oct 30 22:02:25.339156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.339069    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-data" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.339818 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.339228    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-data" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.339818 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.339296    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-credential-socket" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.340370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.339996    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-credential-socket" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.340370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.340197    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-data" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.340559 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.340365    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-certs" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.340559 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.340523    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istiod-ca-cert\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.340674 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.340559    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-certs" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.340674 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.340568    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-token\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-token\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.340674 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.340621    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-podinfo\") pod \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\" (UID: \"11adea8b-7c70-46c4-84cd-d3ed86fcd343\") "
Oct 30 22:02:25.341332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341291    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341323    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341340    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsmbv\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-kube-api-access-gsmbv\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341372    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341387    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341405    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341419    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341433    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341448    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341462    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341475    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18e6ea2-6992-427d-8c0f-6244c76052d6-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341489    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341503    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd7217ba-460e-45c6-804b-70ebd50c3d38-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341517    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.341601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.341531    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9680477d-1d4e-45b5-b96b-f20a0044fd3e-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.347269 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.347236    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-credential-socket" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.348997 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.347797    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istiod-ca-cert" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:25.348997 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.348470    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-socket" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.349803 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.349766    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-token" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.349966 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.349853    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-token" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.350252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.350216    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-kube-api-access-7ws2m" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "kube-api-access-7ws2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.350400 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.350367    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-socket" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.351746 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.351688    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-envoy" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.395422 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.377464    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-envoy" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.395422 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.377555    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-podinfo" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:25.397938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.397819    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-podinfo" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:25.398459 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.398392    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-kube-api-access-k5j24" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "kube-api-access-k5j24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.400176 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.400127    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-token" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.400595 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.400381    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-podinfo" pod "11adea8b-7c70-46c4-84cd-d3ed86fcd343" (UID: "11adea8b-7c70-46c4-84cd-d3ed86fcd343"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:25.400595 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.400463    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-kube-api-access-qdcw9" pod "fd7217ba-460e-45c6-804b-70ebd50c3d38" (UID: "fd7217ba-460e-45c6-804b-70ebd50c3d38"). InnerVolumeSpecName "kube-api-access-qdcw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:25.400595 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.400545    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-envoy" pod "a18e6ea2-6992-427d-8c0f-6244c76052d6" (UID: "a18e6ea2-6992-427d-8c0f-6244c76052d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:25.442797 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442763    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442797 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442796    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442807    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442818    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442827    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442836    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442844    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442853    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18e6ea2-6992-427d-8c0f-6244c76052d6-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442861    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442871    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdcw9\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-kube-api-access-qdcw9\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442880    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ws2m\" (UniqueName: \"kubernetes.io/projected/11adea8b-7c70-46c4-84cd-d3ed86fcd343-kube-api-access-7ws2m\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442888    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd7217ba-460e-45c6-804b-70ebd50c3d38-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442896    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18e6ea2-6992-427d-8c0f-6244c76052d6-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442906    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5j24\" (UniqueName: \"kubernetes.io/projected/a18e6ea2-6992-427d-8c0f-6244c76052d6-kube-api-access-k5j24\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442915    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd7217ba-460e-45c6-804b-70ebd50c3d38-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.442998 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:25.442923    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11adea8b-7c70-46c4-84cd-d3ed86fcd343-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:25.478978 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:25.478939    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:26.021254 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.021223    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"
Oct 30 22:02:26.021779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.021223    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8" event={"ID":"9680477d-1d4e-45b5-b96b-f20a0044fd3e","Type":"ContainerDied","Data":"32acc6dafaab1a0dd3b86874374fe44d81ae9383fbb7d4927cb1f9382924529b"}
Oct 30 22:02:26.021779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.021386    5817 scope.go:122] "RemoveContainer" containerID="0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad"
Oct 30 22:02:26.022834 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.022810    5817 generic.go:356] "Generic (PLEG): container finished" podID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerID="6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1" exitCode=0
Oct 30 22:02:26.022908 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.022851    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"
Oct 30 22:02:26.022908 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.022894    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" event={"ID":"11adea8b-7c70-46c4-84cd-d3ed86fcd343","Type":"ContainerDied","Data":"6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1"}
Oct 30 22:02:26.023038 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.022943    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l" event={"ID":"11adea8b-7c70-46c4-84cd-d3ed86fcd343","Type":"ContainerDied","Data":"f88880fa446e339694ede4bc376712b5ce39939873ce432133f4cc3074ba0928"}
Oct 30 22:02:26.028720 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.028697    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"
Oct 30 22:02:26.028859 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.028715    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj" event={"ID":"a18e6ea2-6992-427d-8c0f-6244c76052d6","Type":"ContainerDied","Data":"4762d213b5999565e524d2f02b9c1a84e5411a8ad2ba90a07dd573588eaa4a3b"}
Oct 30 22:02:26.030636 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.030610    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s" event={"ID":"fd7217ba-460e-45c6-804b-70ebd50c3d38","Type":"ContainerDied","Data":"f80753d6454d5955d44696ca2b278c14c80f2a2a610ccc8e4a1281f755330e72"}
Oct 30 22:02:26.030792 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.030771    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"
Oct 30 22:02:26.037715 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.037694    5817 scope.go:122] "RemoveContainer" containerID="6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1"
Oct 30 22:02:26.047969 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.047953    5817 scope.go:122] "RemoveContainer" containerID="6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1"
Oct 30 22:02:26.048251 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:26.048233    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1\": container with ID starting with 6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1 not found: ID does not exist" containerID="6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1"
Oct 30 22:02:26.048333 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.048265    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1"} err="failed to get container status \"6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1\": rpc error: code = NotFound desc = could not find container \"6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1\": container with ID starting with 6cb658571a1972caba1bd84280d55d415c4bc25635496e0d02f3eb0721806cc1 not found: ID does not exist"
Oct 30 22:02:26.048333 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.048291    5817 scope.go:122] "RemoveContainer" containerID="28ca5ed569ca1ad44b9319d31535c33152d82e307fda7d6ecb66ab1cd4411de4"
Oct 30 22:02:26.057104 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.057078    5817 scope.go:122] "RemoveContainer" containerID="7769fe88a892751ec9bb095c4097637ee58968ae12b63d13a81dc0f6d756651e"
Oct 30 22:02:26.184503 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.184424    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"]
Oct 30 22:02:26.193831 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.193797    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-86c2s"]
Oct 30 22:02:26.308002 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.307962    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"]
Oct 30 22:02:26.312166 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.312139    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-tkw7l"]
Oct 30 22:02:26.415938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.415905    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"]
Oct 30 22:02:26.420127 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.420101    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-788r8"]
Oct 30 22:02:26.431566 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.431531    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" podUID="df85e97b-dc4d-4b76-8977-0c721046960d" containerName="payload-pre-processing" containerID="cri-o://e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032" gracePeriod=25
Oct 30 22:02:26.527485 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.527451    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"]
Oct 30 22:02:26.531867 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.531840    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bfdf30b9-openshift-default-8486754c49-xdcbj"]
Oct 30 22:02:26.844611 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.844534    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" path="/var/lib/kubelet/pods/11adea8b-7c70-46c4-84cd-d3ed86fcd343/volumes"
Oct 30 22:02:26.844992 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.844977    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" path="/var/lib/kubelet/pods/9680477d-1d4e-45b5-b96b-f20a0044fd3e/volumes"
Oct 30 22:02:26.845542 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.845522    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" path="/var/lib/kubelet/pods/a18e6ea2-6992-427d-8c0f-6244c76052d6/volumes"
Oct 30 22:02:26.846297 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:26.846278    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" path="/var/lib/kubelet/pods/fd7217ba-460e-45c6-804b-70ebd50c3d38/volumes"
Oct 30 22:02:29.203442 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.203394    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main" containerID="cri-o://779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56" gracePeriod=45
Oct 30 22:02:29.207022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.206964    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main" containerID="cri-o://462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469" gracePeriod=45
Oct 30 22:02:29.208181 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.208145    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main" containerID="cri-o://e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328" gracePeriod=45
Oct 30 22:02:29.466925 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.466861    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 22:02:29.525887 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.525865    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 22:02:29.529211 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.529191    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 22:02:29.589314 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589282    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28a09d72-9c45-41d6-b755-d018dc2271cb-kube-api-access-bfz2z\" (UniqueName: \"kubernetes.io/projected/28a09d72-9c45-41d6-b755-d018dc2271cb-kube-api-access-bfz2z\") pod \"28a09d72-9c45-41d6-b755-d018dc2271cb\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589363    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-tmp-dir\") pod \"c289aead-2be5-4573-9f7b-52be0b91b408\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589409    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-tmp-dir\") pod \"28a09d72-9c45-41d6-b755-d018dc2271cb\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589483    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-home\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-home\") pod \"28a09d72-9c45-41d6-b755-d018dc2271cb\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589547    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-home\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-home\") pod \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589581    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-dshm\") pod \"28a09d72-9c45-41d6-b755-d018dc2271cb\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589638    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-model-cache\") pod \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589671    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tmp-dir\") pod \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589670    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-tmp-dir" pod "c289aead-2be5-4573-9f7b-52be0b91b408" (UID: "c289aead-2be5-4573-9f7b-52be0b91b408"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589703    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-dshm\") pod \"c289aead-2be5-4573-9f7b-52be0b91b408\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589747    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tls-certs\" (UniqueName: \"kubernetes.io/secret/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tls-certs\") pod \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589787    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-model-cache\") pod \"c289aead-2be5-4573-9f7b-52be0b91b408\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589820    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-model-cache\") pod \"28a09d72-9c45-41d6-b755-d018dc2271cb\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589857    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-dshm\") pod \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589889    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28a09d72-9c45-41d6-b755-d018dc2271cb-tls-certs\" (UniqueName: \"kubernetes.io/secret/28a09d72-9c45-41d6-b755-d018dc2271cb-tls-certs\") pod \"28a09d72-9c45-41d6-b755-d018dc2271cb\" (UID: \"28a09d72-9c45-41d6-b755-d018dc2271cb\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589926    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-home\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-home\") pod \"c289aead-2be5-4573-9f7b-52be0b91b408\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589978    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c289aead-2be5-4573-9f7b-52be0b91b408-tls-certs\" (UniqueName: \"kubernetes.io/secret/c289aead-2be5-4573-9f7b-52be0b91b408-tls-certs\") pod \"c289aead-2be5-4573-9f7b-52be0b91b408\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590021    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c289aead-2be5-4573-9f7b-52be0b91b408-kube-api-access-579bt\" (UniqueName: \"kubernetes.io/projected/c289aead-2be5-4573-9f7b-52be0b91b408-kube-api-access-579bt\") pod \"c289aead-2be5-4573-9f7b-52be0b91b408\" (UID: \"c289aead-2be5-4573-9f7b-52be0b91b408\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589846    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-home" pod "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" (UID: "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589904    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-tmp-dir" pod "28a09d72-9c45-41d6-b755-d018dc2271cb" (UID: "28a09d72-9c45-41d6-b755-d018dc2271cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.589908    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-model-cache" pod "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" (UID: "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590075    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-home" pod "28a09d72-9c45-41d6-b755-d018dc2271cb" (UID: "28a09d72-9c45-41d6-b755-d018dc2271cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590065    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-kube-api-access-65bf4\" (UniqueName: \"kubernetes.io/projected/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-kube-api-access-65bf4\") pod \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\" (UID: \"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52\") "
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590302    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-model-cache" pod "28a09d72-9c45-41d6-b755-d018dc2271cb" (UID: "28a09d72-9c45-41d6-b755-d018dc2271cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590682    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590704    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590718    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590731    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590744    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590756    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590863    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tmp-dir" pod "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" (UID: "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.590980    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-model-cache" pod "c289aead-2be5-4573-9f7b-52be0b91b408" (UID: "c289aead-2be5-4573-9f7b-52be0b91b408"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.591404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.591303    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-home" pod "c289aead-2be5-4573-9f7b-52be0b91b408" (UID: "c289aead-2be5-4573-9f7b-52be0b91b408"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.593845 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.593271    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-dshm" pod "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" (UID: "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.594646 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.594609    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-dshm" pod "28a09d72-9c45-41d6-b755-d018dc2271cb" (UID: "28a09d72-9c45-41d6-b755-d018dc2271cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.594794 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.594778    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-kube-api-access-65bf4" pod "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" (UID: "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52"). InnerVolumeSpecName "kube-api-access-65bf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:29.595010 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.594978    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tls-certs" pod "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" (UID: "57ecfea3-2e0a-4ead-b994-6bc7aca1aa52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:29.595010 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.594986    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a09d72-9c45-41d6-b755-d018dc2271cb-tls-certs" pod "28a09d72-9c45-41d6-b755-d018dc2271cb" (UID: "28a09d72-9c45-41d6-b755-d018dc2271cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:29.595162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.595056    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a09d72-9c45-41d6-b755-d018dc2271cb-kube-api-access-bfz2z" pod "28a09d72-9c45-41d6-b755-d018dc2271cb" (UID: "28a09d72-9c45-41d6-b755-d018dc2271cb"). InnerVolumeSpecName "kube-api-access-bfz2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:29.595671 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.595650    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c289aead-2be5-4573-9f7b-52be0b91b408-kube-api-access-579bt" pod "c289aead-2be5-4573-9f7b-52be0b91b408" (UID: "c289aead-2be5-4573-9f7b-52be0b91b408"). InnerVolumeSpecName "kube-api-access-579bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:29.595840 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.595811    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-dshm" pod "c289aead-2be5-4573-9f7b-52be0b91b408" (UID: "c289aead-2be5-4573-9f7b-52be0b91b408"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:29.596667 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.596640    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c289aead-2be5-4573-9f7b-52be0b91b408-tls-certs" pod "c289aead-2be5-4573-9f7b-52be0b91b408" (UID: "c289aead-2be5-4573-9f7b-52be0b91b408"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:02:29.692496 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692464    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfz2z\" (UniqueName: \"kubernetes.io/projected/28a09d72-9c45-41d6-b755-d018dc2271cb-kube-api-access-bfz2z\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692496 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692494    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a09d72-9c45-41d6-b755-d018dc2271cb-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692508    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692520    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692531    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692543    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692554    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692567    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a09d72-9c45-41d6-b755-d018dc2271cb-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692578    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289aead-2be5-4573-9f7b-52be0b91b408-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692589    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289aead-2be5-4573-9f7b-52be0b91b408-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692601    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-579bt\" (UniqueName: \"kubernetes.io/projected/c289aead-2be5-4573-9f7b-52be0b91b408-kube-api-access-579bt\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:29.692714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:29.692616    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65bf4\" (UniqueName: \"kubernetes.io/projected/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52-kube-api-access-65bf4\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.056592 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.056558    5817 generic.go:356] "Generic (PLEG): container finished" podID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerID="779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56" exitCode=0
Oct 30 22:02:30.056779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.056602    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"
Oct 30 22:02:30.056779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.056639    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" event={"ID":"28a09d72-9c45-41d6-b755-d018dc2271cb","Type":"ContainerDied","Data":"779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56"}
Oct 30 22:02:30.056779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.056686    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz" event={"ID":"28a09d72-9c45-41d6-b755-d018dc2271cb","Type":"ContainerDied","Data":"c76105eab21d70073f83ea3d4e12467f898527e0170988e741cb661384532dfe"}
Oct 30 22:02:30.056779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.056710    5817 scope.go:122] "RemoveContainer" containerID="779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56"
Oct 30 22:02:30.058395 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.058214    5817 generic.go:356] "Generic (PLEG): container finished" podID="c289aead-2be5-4573-9f7b-52be0b91b408" containerID="462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469" exitCode=0
Oct 30 22:02:30.058395 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.058264    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"
Oct 30 22:02:30.058395 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.058324    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" event={"ID":"c289aead-2be5-4573-9f7b-52be0b91b408","Type":"ContainerDied","Data":"462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469"}
Oct 30 22:02:30.058395 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.058383    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d" event={"ID":"c289aead-2be5-4573-9f7b-52be0b91b408","Type":"ContainerDied","Data":"067dee3c8dc59ebe2619292f0531e37929ec63aae6785e792b5549df6e1501a8"}
Oct 30 22:02:30.059738 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.059716    5817 generic.go:356] "Generic (PLEG): container finished" podID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerID="e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328" exitCode=0
Oct 30 22:02:30.059846 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.059754    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" event={"ID":"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52","Type":"ContainerDied","Data":"e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328"}
Oct 30 22:02:30.059846 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.059782    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6" event={"ID":"57ecfea3-2e0a-4ead-b994-6bc7aca1aa52","Type":"ContainerDied","Data":"d93a08fc55f28249599227f43aaff574cc1b74e10565c37465143cef0c1c2f60"}
Oct 30 22:02:30.059846 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.059837    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"
Oct 30 22:02:30.074128 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.074070    5817 scope.go:122] "RemoveContainer" containerID="779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56"
Oct 30 22:02:30.074390 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:30.074336    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56\": container with ID starting with 779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56 not found: ID does not exist" containerID="779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56"
Oct 30 22:02:30.074498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.074399    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56"} err="failed to get container status \"779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56\": rpc error: code = NotFound desc = could not find container \"779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56\": container with ID starting with 779f3e8de52720387035e6e6c697e8a84903251324044e95756cf55a1eb97d56 not found: ID does not exist"
Oct 30 22:02:30.074498 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.074425    5817 scope.go:122] "RemoveContainer" containerID="462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469"
Oct 30 22:02:30.083673 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.083656    5817 scope.go:122] "RemoveContainer" containerID="462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469"
Oct 30 22:02:30.083918 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:30.083901    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469\": container with ID starting with 462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469 not found: ID does not exist" containerID="462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469"
Oct 30 22:02:30.083957 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.083925    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469"} err="failed to get container status \"462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469\": rpc error: code = NotFound desc = could not find container \"462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469\": container with ID starting with 462bb04109c500e15052419e9fb0598a18598d56db2f4e7a27b6d74391a0e469 not found: ID does not exist"
Oct 30 22:02:30.083957 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.083941    5817 scope.go:122] "RemoveContainer" containerID="e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328"
Oct 30 22:02:30.092249 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.092230    5817 scope.go:122] "RemoveContainer" containerID="e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328"
Oct 30 22:02:30.092546 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:30.092526    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328\": container with ID starting with e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328 not found: ID does not exist" containerID="e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328"
Oct 30 22:02:30.092597 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.092551    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328"} err="failed to get container status \"e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328\": rpc error: code = NotFound desc = could not find container \"e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328\": container with ID starting with e39ee7e79a4880ffd467365098d90296cb4b52f22c8ee29cb8b4db4bc632a328 not found: ID does not exist"
Oct 30 22:02:30.212182 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.212154    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"]
Oct 30 22:02:30.218862 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.218840    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-991335/premium-w5-991335-kserve-7f6579d59f-8gldz"]
Oct 30 22:02:30.330132 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.330106    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"]
Oct 30 22:02:30.341664 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.341641    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-991335/simulator-w5-991335-kserve-7d6df54b78-dbkm6"]
Oct 30 22:02:30.452945 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.452917    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"]
Oct 30 22:02:30.462375 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.459100    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-991335/distinct-w5-991335-kserve-68ddbd7fcd-wdk2d"]
Oct 30 22:02:30.551948 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.551920    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 22:02:30.602656 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602595    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-podinfo\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.602656 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602648    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-credential-socket\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.602814 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602673    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-data\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.602814 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602710    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-certs\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.602814 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602735    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istiod-ca-cert\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.602814 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602768    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-token\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-token\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.603017 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602814    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-socket\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.603017 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602847    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-envoy\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.603017 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.602876    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-kube-api-access-8q2t9\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-kube-api-access-8q2t9\") pod \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\" (UID: \"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3\") "
Oct 30 22:02:30.603197 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603011    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-data" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:30.603197 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603036    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-credential-socket" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:30.603197 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603045    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-certs" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:30.603197 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603110    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istiod-ca-cert" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:30.603443 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603420    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-socket" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:30.603626 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603601    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.603745 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603629    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.603745 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603646    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.603745 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603661    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.603745 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.603673    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.604904 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.604878    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-podinfo" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:30.605065 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.605048    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-kube-api-access-8q2t9" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "kube-api-access-8q2t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:30.605168 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.605146    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-envoy" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:30.605203 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.605163    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-token" pod "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" (UID: "eb8a0fb2-6236-41c4-9fde-34d97a9a71d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:30.705074 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.705049    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.705074 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.705073    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.705228 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.705083    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q2t9\" (UniqueName: \"kubernetes.io/projected/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-kube-api-access-8q2t9\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.705228 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.705092    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:30.843724 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.843695    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" path="/var/lib/kubelet/pods/28a09d72-9c45-41d6-b755-d018dc2271cb/volumes"
Oct 30 22:02:30.844144 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.844130    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" path="/var/lib/kubelet/pods/57ecfea3-2e0a-4ead-b994-6bc7aca1aa52/volumes"
Oct 30 22:02:30.844531 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:30.844518    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" path="/var/lib/kubelet/pods/c289aead-2be5-4573-9f7b-52be0b91b408/volumes"
Oct 30 22:02:31.067022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.066989    5817 generic.go:356] "Generic (PLEG): container finished" podID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerID="c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17" exitCode=0
Oct 30 22:02:31.067184 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.067040    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"
Oct 30 22:02:31.067184 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.067067    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" event={"ID":"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3","Type":"ContainerDied","Data":"c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17"}
Oct 30 22:02:31.067184 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.067113    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b" event={"ID":"eb8a0fb2-6236-41c4-9fde-34d97a9a71d3","Type":"ContainerDied","Data":"5395c01e845181553d87b0ad614832e81ff8d94fc13dd1a172418b65c94bad0f"}
Oct 30 22:02:31.067184 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.067141    5817 scope.go:122] "RemoveContainer" containerID="c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17"
Oct 30 22:02:31.077193 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.077159    5817 scope.go:122] "RemoveContainer" containerID="c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17"
Oct 30 22:02:31.077458 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:31.077440    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17\": container with ID starting with c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17 not found: ID does not exist" containerID="c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17"
Oct 30 22:02:31.077537 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.077465    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17"} err="failed to get container status \"c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17\": rpc error: code = NotFound desc = could not find container \"c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17\": container with ID starting with c55398ab314d3c525879d36ec5c4e6e0ff0f9ae69b6f9d84322d3a51c374bb17 not found: ID does not exist"
Oct 30 22:02:31.226182 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.226153    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"]
Oct 30 22:02:31.231997 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:31.231972    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-71aab5-openshift-default-6f49bc5f57-t9c8b"]
Oct 30 22:02:32.843133 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:32.843041    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" path="/var/lib/kubelet/pods/eb8a0fb2-6236-41c4-9fde-34d97a9a71d3/volumes"
Oct 30 22:02:34.534913 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:34.534871    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" podUID="df85e97b-dc4d-4b76-8977-0c721046960d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:02:35.543729 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:35.543695    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:35.951225 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:35.950890    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:44.535568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:44.535522    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" podUID="df85e97b-dc4d-4b76-8977-0c721046960d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:02:45.609158 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:45.609115    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:46.338195 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:46.338160    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:02:46.685618 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:46.685591    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 22:02:46.787248 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:46.787217    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df85e97b-dc4d-4b76-8977-0c721046960d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df85e97b-dc4d-4b76-8977-0c721046960d-plugins-config-volume\") pod \"df85e97b-dc4d-4b76-8977-0c721046960d\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") "
Oct 30 22:02:46.787413 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:46.787383    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df85e97b-dc4d-4b76-8977-0c721046960d-kube-api-access-h6sp7\" (UniqueName: \"kubernetes.io/projected/df85e97b-dc4d-4b76-8977-0c721046960d-kube-api-access-h6sp7\") pod \"df85e97b-dc4d-4b76-8977-0c721046960d\" (UID: \"df85e97b-dc4d-4b76-8977-0c721046960d\") "
Oct 30 22:02:46.787599 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:46.787577    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df85e97b-dc4d-4b76-8977-0c721046960d-plugins-config-volume" pod "df85e97b-dc4d-4b76-8977-0c721046960d" (UID: "df85e97b-dc4d-4b76-8977-0c721046960d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:46.787822 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:46.787806    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df85e97b-dc4d-4b76-8977-0c721046960d-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:46.789625 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:46.789596    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df85e97b-dc4d-4b76-8977-0c721046960d-kube-api-access-h6sp7" pod "df85e97b-dc4d-4b76-8977-0c721046960d" (UID: "df85e97b-dc4d-4b76-8977-0c721046960d"). InnerVolumeSpecName "kube-api-access-h6sp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:46.889267 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:46.889231    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6sp7\" (UniqueName: \"kubernetes.io/projected/df85e97b-dc4d-4b76-8977-0c721046960d-kube-api-access-h6sp7\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:47.177869 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.177783    5817 generic.go:356] "Generic (PLEG): container finished" podID="df85e97b-dc4d-4b76-8977-0c721046960d" containerID="e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032" exitCode=0
Oct 30 22:02:47.177869 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.177834    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"
Oct 30 22:02:47.178057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.177868    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" event={"ID":"df85e97b-dc4d-4b76-8977-0c721046960d","Type":"ContainerDied","Data":"e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032"}
Oct 30 22:02:47.178057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.177926    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58" event={"ID":"df85e97b-dc4d-4b76-8977-0c721046960d","Type":"ContainerDied","Data":"64841bd4e367baa93ba9babfc8c88b2ad13430b3c457c5fdcfdadbec7ba392cd"}
Oct 30 22:02:47.178057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.177951    5817 scope.go:122] "RemoveContainer" containerID="e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032"
Oct 30 22:02:47.190274 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.190255    5817 scope.go:122] "RemoveContainer" containerID="e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032"
Oct 30 22:02:47.190625 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:47.190607    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032\": container with ID starting with e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032 not found: ID does not exist" containerID="e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032"
Oct 30 22:02:47.190701 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.190634    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032"} err="failed to get container status \"e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032\": rpc error: code = NotFound desc = could not find container \"e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032\": container with ID starting with e2c5c6197f3064414a3011e51c24d25d097daef22475dd2da044bf62bbc48032 not found: ID does not exist"
Oct 30 22:02:47.353122 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.353088    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"]
Oct 30 22:02:47.358431 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:47.358407    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-991335-6bcfccf6dc-f8d58"]
Oct 30 22:02:48.851544 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:48.851497    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df85e97b-dc4d-4b76-8977-0c721046960d" path="/var/lib/kubelet/pods/df85e97b-dc4d-4b76-8977-0c721046960d/volumes"
Oct 30 22:02:51.352371 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:51.350107    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:52.064995 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:52.064956    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"]
Oct 30 22:02:52.065291 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:52.065233    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy" containerID="cri-o://b58345370de7281cd256639a4dbe45a03dae272c2067b364178b17cd5d1929a1" gracePeriod=30
Oct 30 22:02:53.128654 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:53.126725    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:53.128654 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:53.126838    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:55.625336 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:02:55.625287    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:02:57.304701 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.304661    5817 generic.go:356] "Generic (PLEG): container finished" podID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerID="b58345370de7281cd256639a4dbe45a03dae272c2067b364178b17cd5d1929a1" exitCode=0
Oct 30 22:02:57.305186 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.304739    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" event={"ID":"9d54ac74-962b-4549-8c9b-a2c98f4d3251","Type":"ContainerDied","Data":"b58345370de7281cd256639a4dbe45a03dae272c2067b364178b17cd5d1929a1"}
Oct 30 22:02:57.305186 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.304815    5817 scope.go:122] "RemoveContainer" containerID="41e1149c1a6943960b443842ebbd9b9838247375d37f8e340225d68296b25e03"
Oct 30 22:02:57.306877 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.306852    5817 generic.go:356] "Generic (PLEG): container finished" podID="8972290a-50d8-4ad3-8998-88a670362dae" containerID="18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a" exitCode=0
Oct 30 22:02:57.306989 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.306914    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" event={"ID":"8972290a-50d8-4ad3-8998-88a670362dae","Type":"ContainerDied","Data":"18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a"}
Oct 30 22:02:57.307340 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.307322    5817 scope.go:122] "RemoveContainer" containerID="18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a"
Oct 30 22:02:57.308729 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.308695    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:02:57.308847 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.308769    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:02:57.308847 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.308800    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:02:57.370021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.369993    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 22:02:57.429539 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429504    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-kube-api-access-zq6l2\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-kube-api-access-zq6l2\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.429723 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429597    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-socket\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.429723 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429633    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-certs\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.429723 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429690    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-token\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-token\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.429905 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429738    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-credential-socket\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.429905 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429778    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-podinfo\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.429905 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429849    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-envoy\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.430073 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429941    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-data\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.430073 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.429981    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istiod-ca-cert\") pod \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\" (UID: \"9d54ac74-962b-4549-8c9b-a2c98f4d3251\") "
Oct 30 22:02:57.430217 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.430179    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-certs" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:57.430633 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.430383    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-socket" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:57.430633 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.430576    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.430633 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.430597    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.430633 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.430618    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-credential-socket" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:57.430945 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.430643    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-data" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:57.430945 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.430810    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istiod-ca-cert" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:02:57.433081 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.433041    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-podinfo" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:02:57.433214 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.433132    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-envoy" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:02:57.433680 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.433624    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-kube-api-access-zq6l2" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "kube-api-access-zq6l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:57.433680 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.433657    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-token" pod "9d54ac74-962b-4549-8c9b-a2c98f4d3251" (UID: "9d54ac74-962b-4549-8c9b-a2c98f4d3251"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:02:57.531445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.531280    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.531445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.531309    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.531445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.531319    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.531445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.531327    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.531445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.531336    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.531445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.531374    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d54ac74-962b-4549-8c9b-a2c98f4d3251-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:57.531445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:57.531385    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq6l2\" (UniqueName: \"kubernetes.io/projected/9d54ac74-962b-4549-8c9b-a2c98f4d3251-kube-api-access-zq6l2\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:02:58.320859 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:58.320826    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"
Oct 30 22:02:58.321278 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:58.320867    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl" event={"ID":"9d54ac74-962b-4549-8c9b-a2c98f4d3251","Type":"ContainerDied","Data":"ac4aee690dc77ef4a7a34b65b46b57dc2458e511c47b58fbbe8a4b198af27957"}
Oct 30 22:02:58.321278 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:58.320949    5817 scope.go:122] "RemoveContainer" containerID="b58345370de7281cd256639a4dbe45a03dae272c2067b364178b17cd5d1929a1"
Oct 30 22:02:58.323654 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:58.323626    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" event={"ID":"8972290a-50d8-4ad3-8998-88a670362dae","Type":"ContainerStarted","Data":"81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266"}
Oct 30 22:02:58.523458 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:58.523413    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"]
Oct 30 22:02:58.570180 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:58.570146    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-a2dced-openshift-default-557799c76c-qcztl"]
Oct 30 22:02:58.844674 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:58.844640    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" path="/var/lib/kubelet/pods/9d54ac74-962b-4549-8c9b-a2c98f4d3251/volumes"
Oct 30 22:02:59.121962 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:59.121887    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:02:59.122786 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:59.122761    5817 patch_prober.go:29] interesting pod/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:02:59.122913 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:02:59.122808    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:03:00.121887 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:00.121852    5817 patch_prober.go:29] interesting pod/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:03:00.122320 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:00.121905    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:03:00.808327 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:00.808292    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:03:01.121792 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:01.121695    5817 patch_prober.go:29] interesting pod/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:03:01.121792 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:01.121752    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:03:02.122235 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:02.122201    5817 patch_prober.go:29] interesting pod/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:03:02.122677 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:02.122253    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:03:03.121891 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:03.121849    5817 patch_prober.go:29] interesting pod/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:03:03.122089 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:03.121918    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 22:03:04.126384 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:04.126332    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:03:04.127448 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:04.127425    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:03:05.688440 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:05.688404    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:03:05.934885 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:05.934841    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:03:15.751847 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:15.751810    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:03:16.331973 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:16.331938    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"]
Oct 30 22:03:16.332182 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:16.332144    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" containerID="cri-o://b28c534845a777a744a5c8e6060c6d53dd37c1f49cf617c3d28c78ae599b72f0" gracePeriod=30
Oct 30 22:03:16.338335 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:16.338306    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:03:17.020166 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:17.020130    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"]
Oct 30 22:03:17.147374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:17.147315    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"]
Oct 30 22:03:19.822126 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:19.822088    5817 patch_prober.go:29] interesting pod/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:03:19.822524 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:19.822140    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:03:20.990157 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:20.990120    5817 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9680477d_1d4e_45b5_b96b_f20a0044fd3e.slice/crio-conmon-0a543e2e1e0db75032abe405387149b1f0d529ec8f7fd4e8a17ab373950dfbad.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:03:21.452779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.452750    5817 generic.go:356] "Generic (PLEG): container finished" podID="ff45733c-4a30-43c1-b509-18792c82ec04" containerID="b28c534845a777a744a5c8e6060c6d53dd37c1f49cf617c3d28c78ae599b72f0" exitCode=0
Oct 30 22:03:21.452885 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.452824    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" event={"ID":"ff45733c-4a30-43c1-b509-18792c82ec04","Type":"ContainerDied","Data":"b28c534845a777a744a5c8e6060c6d53dd37c1f49cf617c3d28c78ae599b72f0"}
Oct 30 22:03:21.452885 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.452881    5817 scope.go:122] "RemoveContainer" containerID="38019071fc0d63dce06612171d26127afcc6b1c2595983311f25c9d7ea9ace40"
Oct 30 22:03:21.589279 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.589252    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 22:03:21.623922 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.623866    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-envoy\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.623922 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.623902    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-socket\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.623934    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-certs\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.623971    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-data\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624016    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-credential-socket\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624063    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ff45733c-4a30-43c1-b509-18792c82ec04-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff45733c-4a30-43c1-b509-18792c82ec04-istio-podinfo\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624098    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff45733c-4a30-43c1-b509-18792c82ec04-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff45733c-4a30-43c1-b509-18792c82ec04-istiod-ca-cert\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624425 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624131    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-socket" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:21.624425 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624170    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-istio-token\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-istio-token\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624425 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624231    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-kube-api-access-d2x7l\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-kube-api-access-d2x7l\") pod \"ff45733c-4a30-43c1-b509-18792c82ec04\" (UID: \"ff45733c-4a30-43c1-b509-18792c82ec04\") "
Oct 30 22:03:21.624425 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624289    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-certs" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:21.624425 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624327    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-credential-socket" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:21.624690 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624554    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff45733c-4a30-43c1-b509-18792c82ec04-istiod-ca-cert" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:21.624690 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624589    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-data" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:21.624806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624746    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.624806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624766    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.624806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624776    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff45733c-4a30-43c1-b509-18792c82ec04-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.624806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624785    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.624806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.624797    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.626192 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.626166    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff45733c-4a30-43c1-b509-18792c82ec04-istio-podinfo" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:03:21.626310 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.626176    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-envoy" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:21.626700 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.626670    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-istio-token" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:21.626779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.626698    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-kube-api-access-d2x7l" pod "ff45733c-4a30-43c1-b509-18792c82ec04" (UID: "ff45733c-4a30-43c1-b509-18792c82ec04"). InnerVolumeSpecName "kube-api-access-d2x7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:21.726279 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.726249    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff45733c-4a30-43c1-b509-18792c82ec04-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.726279 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.726275    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.726482 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.726286    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2x7l\" (UniqueName: \"kubernetes.io/projected/ff45733c-4a30-43c1-b509-18792c82ec04-kube-api-access-d2x7l\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:21.726482 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:21.726297    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff45733c-4a30-43c1-b509-18792c82ec04-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:22.021039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.020998    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq" podUID="efc0e828-5e19-4eed-87df-6ba8727d9c73" containerName="payload-processing" containerID="cri-o://1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e" gracePeriod=25
Oct 30 22:03:22.148494 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.148461    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w" podUID="70a9167f-e9f7-4108-876c-22c4784f2c94" containerName="payload-pre-processing" containerID="cri-o://621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4" gracePeriod=25
Oct 30 22:03:22.267627 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.267604    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:03:22.332594 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.332570    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efc0e828-5e19-4eed-87df-6ba8727d9c73-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc0e828-5e19-4eed-87df-6ba8727d9c73-plugins-config-volume\") pod \"efc0e828-5e19-4eed-87df-6ba8727d9c73\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") "
Oct 30 22:03:22.332771 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.332736    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efc0e828-5e19-4eed-87df-6ba8727d9c73-kube-api-access-6cxfn\" (UniqueName: \"kubernetes.io/projected/efc0e828-5e19-4eed-87df-6ba8727d9c73-kube-api-access-6cxfn\") pod \"efc0e828-5e19-4eed-87df-6ba8727d9c73\" (UID: \"efc0e828-5e19-4eed-87df-6ba8727d9c73\") "
Oct 30 22:03:22.333045 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.333015    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efc0e828-5e19-4eed-87df-6ba8727d9c73-plugins-config-volume" pod "efc0e828-5e19-4eed-87df-6ba8727d9c73" (UID: "efc0e828-5e19-4eed-87df-6ba8727d9c73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:22.335818 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.335790    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc0e828-5e19-4eed-87df-6ba8727d9c73-kube-api-access-6cxfn" pod "efc0e828-5e19-4eed-87df-6ba8727d9c73" (UID: "efc0e828-5e19-4eed-87df-6ba8727d9c73"). InnerVolumeSpecName "kube-api-access-6cxfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:22.408917 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.408886    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:03:22.433825 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.433794    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70a9167f-e9f7-4108-876c-22c4784f2c94-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70a9167f-e9f7-4108-876c-22c4784f2c94-plugins-config-volume\") pod \"70a9167f-e9f7-4108-876c-22c4784f2c94\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") "
Oct 30 22:03:22.433989 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.433880    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70a9167f-e9f7-4108-876c-22c4784f2c94-kube-api-access-2szjf\" (UniqueName: \"kubernetes.io/projected/70a9167f-e9f7-4108-876c-22c4784f2c94-kube-api-access-2szjf\") pod \"70a9167f-e9f7-4108-876c-22c4784f2c94\" (UID: \"70a9167f-e9f7-4108-876c-22c4784f2c94\") "
Oct 30 22:03:22.434161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.434142    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cxfn\" (UniqueName: \"kubernetes.io/projected/efc0e828-5e19-4eed-87df-6ba8727d9c73-kube-api-access-6cxfn\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:22.434161 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.434157    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efc0e828-5e19-4eed-87df-6ba8727d9c73-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:22.434301 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.434201    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70a9167f-e9f7-4108-876c-22c4784f2c94-plugins-config-volume" pod "70a9167f-e9f7-4108-876c-22c4784f2c94" (UID: "70a9167f-e9f7-4108-876c-22c4784f2c94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:22.436182 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.436163    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a9167f-e9f7-4108-876c-22c4784f2c94-kube-api-access-2szjf" pod "70a9167f-e9f7-4108-876c-22c4784f2c94" (UID: "70a9167f-e9f7-4108-876c-22c4784f2c94"). InnerVolumeSpecName "kube-api-access-2szjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:22.459359 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.459324    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"
Oct 30 22:03:22.459468 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.459356    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2" event={"ID":"ff45733c-4a30-43c1-b509-18792c82ec04","Type":"ContainerDied","Data":"bc6af06dd1cbc060bf49728eb89b5e27cd257fbe3e3b2bb30b80c7ff7fbb33e1"}
Oct 30 22:03:22.459468 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.459429    5817 scope.go:122] "RemoveContainer" containerID="b28c534845a777a744a5c8e6060c6d53dd37c1f49cf617c3d28c78ae599b72f0"
Oct 30 22:03:22.460734 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.460711    5817 generic.go:356] "Generic (PLEG): container finished" podID="70a9167f-e9f7-4108-876c-22c4784f2c94" containerID="621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4" exitCode=0
Oct 30 22:03:22.460800 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.460762    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"
Oct 30 22:03:22.460800 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.460787    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w" event={"ID":"70a9167f-e9f7-4108-876c-22c4784f2c94","Type":"ContainerDied","Data":"621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4"}
Oct 30 22:03:22.460880 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.460833    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w" event={"ID":"70a9167f-e9f7-4108-876c-22c4784f2c94","Type":"ContainerDied","Data":"b3e68e7a81966812ab22bd6457e35c770aec54583c9419ebed3bb781111c982c"}
Oct 30 22:03:22.462198 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.462167    5817 generic.go:356] "Generic (PLEG): container finished" podID="efc0e828-5e19-4eed-87df-6ba8727d9c73" containerID="1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e" exitCode=0
Oct 30 22:03:22.462290 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.462218    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"
Oct 30 22:03:22.462290 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.462244    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq" event={"ID":"efc0e828-5e19-4eed-87df-6ba8727d9c73","Type":"ContainerDied","Data":"1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e"}
Oct 30 22:03:22.462290 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.462272    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq" event={"ID":"efc0e828-5e19-4eed-87df-6ba8727d9c73","Type":"ContainerDied","Data":"019c1a77e6783a13ad1898bc3fde4f8d7bcedcdec53633fd7b63d24222b8aa04"}
Oct 30 22:03:22.470207 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.470190    5817 scope.go:122] "RemoveContainer" containerID="621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4"
Oct 30 22:03:22.479262 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.479242    5817 scope.go:122] "RemoveContainer" containerID="621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4"
Oct 30 22:03:22.479555 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:22.479534    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4\": container with ID starting with 621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4 not found: ID does not exist" containerID="621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4"
Oct 30 22:03:22.479612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.479562    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4"} err="failed to get container status \"621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4\": rpc error: code = NotFound desc = could not find container \"621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4\": container with ID starting with 621e0b847bb7194f7a68317b8955f7a14ac5233876d023768a29e2c2fc142fe4 not found: ID does not exist"
Oct 30 22:03:22.479612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.479579    5817 scope.go:122] "RemoveContainer" containerID="1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e"
Oct 30 22:03:22.487073 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.487057    5817 scope.go:122] "RemoveContainer" containerID="1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e"
Oct 30 22:03:22.487315 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:22.487292    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e\": container with ID starting with 1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e not found: ID does not exist" containerID="1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e"
Oct 30 22:03:22.487409 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.487324    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e"} err="failed to get container status \"1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e\": rpc error: code = NotFound desc = could not find container \"1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e\": container with ID starting with 1959f5aa6ea452c74bbdbeb4c1ecede81cac3ed82eb457c6d357a603b5cc911e not found: ID does not exist"
Oct 30 22:03:22.535508 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.535482    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70a9167f-e9f7-4108-876c-22c4784f2c94-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:22.535508 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.535508    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2szjf\" (UniqueName: \"kubernetes.io/projected/70a9167f-e9f7-4108-876c-22c4784f2c94-kube-api-access-2szjf\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:22.625766 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.625702    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"]
Oct 30 22:03:22.630857 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.630833    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-28c192-949776c8c-jpc5w"]
Oct 30 22:03:22.733662 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.733634    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"]
Oct 30 22:03:22.741131 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.741108    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-28c192-797ddf4b4d-npqhq"]
Oct 30 22:03:22.842975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.842943    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70a9167f-e9f7-4108-876c-22c4784f2c94" path="/var/lib/kubelet/pods/70a9167f-e9f7-4108-876c-22c4784f2c94/volumes"
Oct 30 22:03:22.843358 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.843335    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efc0e828-5e19-4eed-87df-6ba8727d9c73" path="/var/lib/kubelet/pods/efc0e828-5e19-4eed-87df-6ba8727d9c73/volumes"
Oct 30 22:03:22.847911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.847887    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"]
Oct 30 22:03:22.852863 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:22.852833    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-991335-openshift-default-6c6b8d69bc-6c8h2"]
Oct 30 22:03:24.842486 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:24.842451    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" path="/var/lib/kubelet/pods/ff45733c-4a30-43c1-b509-18792c82ec04/volumes"
Oct 30 22:03:46.337761 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:46.337718    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:03:54.966738 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:54.966700    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"]
Oct 30 22:03:54.967191 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:54.966894    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k" podUID="c8e358dd-6be9-4c08-a230-f4c9a377bb3a" containerName="maas-api" containerID="cri-o://f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21" gracePeriod=30
Oct 30 22:03:55.228566 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.228505    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:03:55.271770 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.271741    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-metrics-certs\") pod \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") "
Oct 30 22:03:55.271916 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.271775    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-openshift-service-ca\") pod \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") "
Oct 30 22:03:55.271916 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.271870    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls\") pod \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") "
Oct 30 22:03:55.271916 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.271906    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-kube-api-access-crzsd\" (UniqueName: \"kubernetes.io/projected/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-kube-api-access-crzsd\") pod \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\" (UID: \"c8e358dd-6be9-4c08-a230-f4c9a377bb3a\") "
Oct 30 22:03:55.272170 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.272141    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-openshift-service-ca" pod "c8e358dd-6be9-4c08-a230-f4c9a377bb3a" (UID: "c8e358dd-6be9-4c08-a230-f4c9a377bb3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:03:55.272299 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.272267    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.273907 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.273878    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls" pod "c8e358dd-6be9-4c08-a230-f4c9a377bb3a" (UID: "c8e358dd-6be9-4c08-a230-f4c9a377bb3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:55.273997 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.273940    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-metrics-certs" pod "c8e358dd-6be9-4c08-a230-f4c9a377bb3a" (UID: "c8e358dd-6be9-4c08-a230-f4c9a377bb3a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:03:55.273997 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.273949    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-kube-api-access-crzsd" pod "c8e358dd-6be9-4c08-a230-f4c9a377bb3a" (UID: "c8e358dd-6be9-4c08-a230-f4c9a377bb3a"). InnerVolumeSpecName "kube-api-access-crzsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:55.373852 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.373814    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.373852 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.373854    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.374049 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.373872    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crzsd\" (UniqueName: \"kubernetes.io/projected/c8e358dd-6be9-4c08-a230-f4c9a377bb3a-kube-api-access-crzsd\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:03:55.632377 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.632260    5817 generic.go:356] "Generic (PLEG): container finished" podID="c8e358dd-6be9-4c08-a230-f4c9a377bb3a" containerID="f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21" exitCode=0
Oct 30 22:03:55.632377 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.632312    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"
Oct 30 22:03:55.632572 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.632377    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k" event={"ID":"c8e358dd-6be9-4c08-a230-f4c9a377bb3a","Type":"ContainerDied","Data":"f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21"}
Oct 30 22:03:55.632572 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.632434    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k" event={"ID":"c8e358dd-6be9-4c08-a230-f4c9a377bb3a","Type":"ContainerDied","Data":"b4d14fcecbf8e17583a4103e2a4229d15dd0015f118d38e0f0778459db745163"}
Oct 30 22:03:55.632572 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.632461    5817 scope.go:122] "RemoveContainer" containerID="f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21"
Oct 30 22:03:55.643111 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.643093    5817 scope.go:122] "RemoveContainer" containerID="f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21"
Oct 30 22:03:55.643401 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:03:55.643374    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21\": container with ID starting with f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21 not found: ID does not exist" containerID="f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21"
Oct 30 22:03:55.643467 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.643411    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21"} err="failed to get container status \"f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21\": rpc error: code = NotFound desc = could not find container \"f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21\": container with ID starting with f43477ef6f7891dbd7ef0d695f4194b21d00c10b934e5aa1c610c7487c82ba21 not found: ID does not exist"
Oct 30 22:03:55.789900 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.789864    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"]
Oct 30 22:03:55.837271 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:55.837235    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-28c192-74497557c6-bp89k"]
Oct 30 22:03:56.842314 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:03:56.842272    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8e358dd-6be9-4c08-a230-f4c9a377bb3a" path="/var/lib/kubelet/pods/c8e358dd-6be9-4c08-a230-f4c9a377bb3a/volumes"
Oct 30 22:04:05.361424 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:05.359644    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"]
Oct 30 22:04:16.337530 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:04:16.337490    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:04:20.551710 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.551669    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main" containerID="cri-o://85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65" gracePeriod=45
Oct 30 22:04:20.806526 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.806466    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 22:04:20.823628 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.823600    5817 generic.go:356] "Generic (PLEG): container finished" podID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerID="85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65" exitCode=0
Oct 30 22:04:20.823759 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.823637    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"
Oct 30 22:04:20.823759 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.823675    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" event={"ID":"8b5efc58-2ff8-4a72-a352-925e7bbe3a24","Type":"ContainerDied","Data":"85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65"}
Oct 30 22:04:20.823759 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.823713    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5" event={"ID":"8b5efc58-2ff8-4a72-a352-925e7bbe3a24","Type":"ContainerDied","Data":"57df3b38b0591e2ceaa59907beec03eafb51f327b8f5249b5742891dc04b19f3"}
Oct 30 22:04:20.823759 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.823744    5817 scope.go:122] "RemoveContainer" containerID="85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65"
Oct 30 22:04:20.833224 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.833205    5817 scope.go:122] "RemoveContainer" containerID="85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65"
Oct 30 22:04:20.833500 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:04:20.833481    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65\": container with ID starting with 85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65 not found: ID does not exist" containerID="85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65"
Oct 30 22:04:20.833573 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.833508    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65"} err="failed to get container status \"85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65\": rpc error: code = NotFound desc = could not find container \"85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65\": container with ID starting with 85ba3330468fca60f8a8cb8ba549055dc61552a29852f60aef9be51c763ebc65 not found: ID does not exist"
Oct 30 22:04:20.958621 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.958591    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tls-certs\") pod \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") "
Oct 30 22:04:20.958808 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.958699    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-model-cache\") pod \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") "
Oct 30 22:04:20.958886 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.958817    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-kube-api-access-4qqg4\" (UniqueName: \"kubernetes.io/projected/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-kube-api-access-4qqg4\") pod \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") "
Oct 30 22:04:20.958946 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.958881    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-home\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-home\") pod \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") "
Oct 30 22:04:20.959006 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.958969    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-model-cache" pod "8b5efc58-2ff8-4a72-a352-925e7bbe3a24" (UID: "8b5efc58-2ff8-4a72-a352-925e7bbe3a24"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:20.959064 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.959008    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tmp-dir\") pod \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") "
Oct 30 22:04:20.959122 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.959084    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-dshm\") pod \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\" (UID: \"8b5efc58-2ff8-4a72-a352-925e7bbe3a24\") "
Oct 30 22:04:20.959184 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.959118    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-home" pod "8b5efc58-2ff8-4a72-a352-925e7bbe3a24" (UID: "8b5efc58-2ff8-4a72-a352-925e7bbe3a24"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:20.959301 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.959271    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tmp-dir" pod "8b5efc58-2ff8-4a72-a352-925e7bbe3a24" (UID: "8b5efc58-2ff8-4a72-a352-925e7bbe3a24"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:20.959647 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.959627    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:20.959724 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.959650    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:20.959724 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.959660    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:20.962176 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.962154    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tls-certs" pod "8b5efc58-2ff8-4a72-a352-925e7bbe3a24" (UID: "8b5efc58-2ff8-4a72-a352-925e7bbe3a24"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:04:20.962332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.962169    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-kube-api-access-4qqg4" pod "8b5efc58-2ff8-4a72-a352-925e7bbe3a24" (UID: "8b5efc58-2ff8-4a72-a352-925e7bbe3a24"). InnerVolumeSpecName "kube-api-access-4qqg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:20.962438 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:20.962340    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-dshm" pod "8b5efc58-2ff8-4a72-a352-925e7bbe3a24" (UID: "8b5efc58-2ff8-4a72-a352-925e7bbe3a24"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:21.061246 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:21.061166    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:21.061246 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:21.061196    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:21.061246 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:21.061208    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qqg4\" (UniqueName: \"kubernetes.io/projected/8b5efc58-2ff8-4a72-a352-925e7bbe3a24-kube-api-access-4qqg4\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:21.255004 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:21.254932    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"]
Oct 30 22:04:21.270824 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:21.270780    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-ed76b8/distinct-2-w2-ed76b8-kserve-7d8d55f4c8-fr5m5"]
Oct 30 22:04:22.842438 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:22.842406    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" path="/var/lib/kubelet/pods/8b5efc58-2ff8-4a72-a352-925e7bbe3a24/volumes"
Oct 30 22:04:46.338385 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:04:46.338321    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:04:46.462721 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:46.462688    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"]
Oct 30 22:04:46.462938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:46.462899    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy" containerID="cri-o://81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266" gracePeriod=30
Oct 30 22:04:48.397037 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.397002    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"]
Oct 30 22:04:48.397477 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.397183    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v" podUID="5930a3d6-c4e4-4518-bee4-de0e1340a990" containerName="maas-api" containerID="cri-o://be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b" gracePeriod=30
Oct 30 22:04:48.655231 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.655174    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 22:04:48.734640 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.734608    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-metrics-certs\") pod \"5930a3d6-c4e4-4518-bee4-de0e1340a990\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") "
Oct 30 22:04:48.734793 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.734682    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5930a3d6-c4e4-4518-bee4-de0e1340a990-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5930a3d6-c4e4-4518-bee4-de0e1340a990-openshift-service-ca\") pod \"5930a3d6-c4e4-4518-bee4-de0e1340a990\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") "
Oct 30 22:04:48.734793 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.734726    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5930a3d6-c4e4-4518-bee4-de0e1340a990-kube-api-access-xdkb8\" (UniqueName: \"kubernetes.io/projected/5930a3d6-c4e4-4518-bee4-de0e1340a990-kube-api-access-xdkb8\") pod \"5930a3d6-c4e4-4518-bee4-de0e1340a990\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") "
Oct 30 22:04:48.734906 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.734799    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls\") pod \"5930a3d6-c4e4-4518-bee4-de0e1340a990\" (UID: \"5930a3d6-c4e4-4518-bee4-de0e1340a990\") "
Oct 30 22:04:48.735037 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.735014    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5930a3d6-c4e4-4518-bee4-de0e1340a990-openshift-service-ca" pod "5930a3d6-c4e4-4518-bee4-de0e1340a990" (UID: "5930a3d6-c4e4-4518-bee4-de0e1340a990"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:48.735233 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.735217    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5930a3d6-c4e4-4518-bee4-de0e1340a990-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:48.736779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.736750    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-metrics-certs" pod "5930a3d6-c4e4-4518-bee4-de0e1340a990" (UID: "5930a3d6-c4e4-4518-bee4-de0e1340a990"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:04:48.736884 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.736797    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5930a3d6-c4e4-4518-bee4-de0e1340a990-kube-api-access-xdkb8" pod "5930a3d6-c4e4-4518-bee4-de0e1340a990" (UID: "5930a3d6-c4e4-4518-bee4-de0e1340a990"). InnerVolumeSpecName "kube-api-access-xdkb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:48.736884 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.736830    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls" pod "5930a3d6-c4e4-4518-bee4-de0e1340a990" (UID: "5930a3d6-c4e4-4518-bee4-de0e1340a990"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:04:48.836832 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.836799    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdkb8\" (UniqueName: \"kubernetes.io/projected/5930a3d6-c4e4-4518-bee4-de0e1340a990-kube-api-access-xdkb8\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:48.836832 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.836834    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:48.837017 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.836943    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5930a3d6-c4e4-4518-bee4-de0e1340a990-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:48.969945 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.969898    5817 generic.go:356] "Generic (PLEG): container finished" podID="5930a3d6-c4e4-4518-bee4-de0e1340a990" containerID="be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b" exitCode=0
Oct 30 22:04:48.970155 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.969956    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"
Oct 30 22:04:48.970155 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.969981    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v" event={"ID":"5930a3d6-c4e4-4518-bee4-de0e1340a990","Type":"ContainerDied","Data":"be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b"}
Oct 30 22:04:48.970155 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.970048    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v" event={"ID":"5930a3d6-c4e4-4518-bee4-de0e1340a990","Type":"ContainerDied","Data":"6a95580e3a5e72b534ab2ea2df0a71f1643d72d4d5cc3ddaec675db0382d3b9c"}
Oct 30 22:04:48.970155 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.970074    5817 scope.go:122] "RemoveContainer" containerID="be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b"
Oct 30 22:04:48.986398 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.986338    5817 scope.go:122] "RemoveContainer" containerID="be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b"
Oct 30 22:04:48.986890 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:04:48.986865    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b\": container with ID starting with be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b not found: ID does not exist" containerID="be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b"
Oct 30 22:04:48.986978 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:48.986902    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b"} err="failed to get container status \"be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b\": rpc error: code = NotFound desc = could not find container \"be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b\": container with ID starting with be4d9d1ffaf10a73f2cfe39ffd458b65cd173fa9be6a52fb120663c1eee7365b not found: ID does not exist"
Oct 30 22:04:49.128058 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:49.128032    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"]
Oct 30 22:04:49.129066 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:49.129035    5817 patch_prober.go:29] interesting pod/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:04:49.129170 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:49.129084    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:04:49.132308 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:49.132284    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ed76b8-668c64ff89-mbj9v"]
Oct 30 22:04:50.842642 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:50.842606    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5930a3d6-c4e4-4518-bee4-de0e1340a990" path="/var/lib/kubelet/pods/5930a3d6-c4e4-4518-bee4-de0e1340a990/volumes"
Oct 30 22:04:51.717885 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.717860    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:04:51.776454 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776378    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-envoy\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776454 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776436    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8972290a-50d8-4ad3-8998-88a670362dae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8972290a-50d8-4ad3-8998-88a670362dae-istiod-ca-cert\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776479    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-credential-socket\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776516    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-certs\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776544    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-socket\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776592    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-data\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776634    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8972290a-50d8-4ad3-8998-88a670362dae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8972290a-50d8-4ad3-8998-88a670362dae-istio-podinfo\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776688    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-kube-api-access-dsjhv\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-kube-api-access-dsjhv\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776770    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-istio-token\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-istio-token\") pod \"8972290a-50d8-4ad3-8998-88a670362dae\" (UID: \"8972290a-50d8-4ad3-8998-88a670362dae\") "
Oct 30 22:04:51.776911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776816    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-certs" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:51.776911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776837    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-credential-socket" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:51.776911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776870    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-socket" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:51.777148 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776915    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8972290a-50d8-4ad3-8998-88a670362dae-istiod-ca-cert" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:51.777148 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.776929    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-data" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:51.777271 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.777253    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.777332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.777277    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8972290a-50d8-4ad3-8998-88a670362dae-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.777332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.777294    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.777332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.777308    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.777332 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.777322    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.778763 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.778740    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-envoy" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:51.778935 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.778911    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-istio-token" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:51.779273 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.779255    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8972290a-50d8-4ad3-8998-88a670362dae-istio-podinfo" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:51.779336 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.779288    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-kube-api-access-dsjhv" pod "8972290a-50d8-4ad3-8998-88a670362dae" (UID: "8972290a-50d8-4ad3-8998-88a670362dae"). InnerVolumeSpecName "kube-api-access-dsjhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:51.878793 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.878763    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8972290a-50d8-4ad3-8998-88a670362dae-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.878793 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.878789    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsjhv\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-kube-api-access-dsjhv\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.879179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.878808    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8972290a-50d8-4ad3-8998-88a670362dae-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:51.879179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:51.878818    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8972290a-50d8-4ad3-8998-88a670362dae-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:04:52.000864 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.000831    5817 generic.go:356] "Generic (PLEG): container finished" podID="8972290a-50d8-4ad3-8998-88a670362dae" containerID="81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266" exitCode=0
Oct 30 22:04:52.001034 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.000880    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"
Oct 30 22:04:52.001034 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.000918    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" event={"ID":"8972290a-50d8-4ad3-8998-88a670362dae","Type":"ContainerDied","Data":"81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266"}
Oct 30 22:04:52.001034 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.000972    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv" event={"ID":"8972290a-50d8-4ad3-8998-88a670362dae","Type":"ContainerDied","Data":"5d4e9023ad16a9b18df4e38d1b34456e651d486e7845780098fad052a4b2e854"}
Oct 30 22:04:52.001034 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.000995    5817 scope.go:122] "RemoveContainer" containerID="81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266"
Oct 30 22:04:52.011056 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.011001    5817 scope.go:122] "RemoveContainer" containerID="18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a"
Oct 30 22:04:52.020447 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.020429    5817 scope.go:122] "RemoveContainer" containerID="81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266"
Oct 30 22:04:52.020757 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:04:52.020739    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266\": container with ID starting with 81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266 not found: ID does not exist" containerID="81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266"
Oct 30 22:04:52.020806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.020767    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266"} err="failed to get container status \"81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266\": rpc error: code = NotFound desc = could not find container \"81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266\": container with ID starting with 81b6f1443ad48cb4515e82158cccc4e81a7de8151f8138d13bc5fbd5fcae6266 not found: ID does not exist"
Oct 30 22:04:52.020806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.020786    5817 scope.go:122] "RemoveContainer" containerID="18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a"
Oct 30 22:04:52.021047 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:04:52.021025    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a\": container with ID starting with 18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a not found: ID does not exist" containerID="18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a"
Oct 30 22:04:52.021090 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.021058    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a"} err="failed to get container status \"18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a\": rpc error: code = NotFound desc = could not find container \"18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a\": container with ID starting with 18d1cf46294ccc17ef132cc77ee7f3662d72dbb89754e8a746d0b51b077bce9a not found: ID does not exist"
Oct 30 22:04:52.159659 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.159586    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"]
Oct 30 22:04:52.163837 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.163811    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-87eb6a-openshift-default-5d4b9cbc69-8pjpv"]
Oct 30 22:04:52.843252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:52.843210    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8972290a-50d8-4ad3-8998-88a670362dae" path="/var/lib/kubelet/pods/8972290a-50d8-4ad3-8998-88a670362dae/volumes"
Oct 30 22:04:59.575750 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:59.575708    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"]
Oct 30 22:04:59.576266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:04:59.575927    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy" containerID="cri-o://e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e" gracePeriod=30
Oct 30 22:05:04.821164 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.821137    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 22:05:04.903878 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.903806    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-data\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.903878 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.903856    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20f96548-9ebe-4495-8aea-c3baf811f695-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f96548-9ebe-4495-8aea-c3baf811f695-istio-podinfo\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.903878 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.903874    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-socket\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.904107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.903897    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-kube-api-access-25fq2\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-kube-api-access-25fq2\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.904107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.903921    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-credential-socket\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.904107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.903985    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-envoy\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.904107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904024    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-istio-token\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-istio-token\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.904107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904093    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-data" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:04.904427 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904115    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20f96548-9ebe-4495-8aea-c3baf811f695-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f96548-9ebe-4495-8aea-c3baf811f695-istiod-ca-cert\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.904427 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904152    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-certs\") pod \"20f96548-9ebe-4495-8aea-c3baf811f695\" (UID: \"20f96548-9ebe-4495-8aea-c3baf811f695\") "
Oct 30 22:05:04.904427 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904204    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-socket" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:04.904427 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904303    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-credential-socket" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:04.904665 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904548    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:04.904665 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904565    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-certs" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:04.904665 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904571    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:04.904665 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904587    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f96548-9ebe-4495-8aea-c3baf811f695-istiod-ca-cert" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:04.904665 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.904596    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:04.906019 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.905998    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20f96548-9ebe-4495-8aea-c3baf811f695-istio-podinfo" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:05:04.906188 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.906170    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-envoy" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:05:04.906528 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.906512    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-kube-api-access-25fq2" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "kube-api-access-25fq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:04.906610 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:04.906523    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-istio-token" pod "20f96548-9ebe-4495-8aea-c3baf811f695" (UID: "20f96548-9ebe-4495-8aea-c3baf811f695"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:05.005406 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.005342    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:05.005406 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.005406    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:05.005649 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.005421    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20f96548-9ebe-4495-8aea-c3baf811f695-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:05.005649 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.005435    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20f96548-9ebe-4495-8aea-c3baf811f695-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:05.005649 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.005449    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20f96548-9ebe-4495-8aea-c3baf811f695-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:05.005649 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.005464    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25fq2\" (UniqueName: \"kubernetes.io/projected/20f96548-9ebe-4495-8aea-c3baf811f695-kube-api-access-25fq2\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:05.068723 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.068691    5817 generic.go:356] "Generic (PLEG): container finished" podID="20f96548-9ebe-4495-8aea-c3baf811f695" containerID="e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e" exitCode=0
Oct 30 22:05:05.068896 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.068743    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"
Oct 30 22:05:05.068896 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.068769    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" event={"ID":"20f96548-9ebe-4495-8aea-c3baf811f695","Type":"ContainerDied","Data":"e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e"}
Oct 30 22:05:05.068896 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.068820    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5" event={"ID":"20f96548-9ebe-4495-8aea-c3baf811f695","Type":"ContainerDied","Data":"a5d5bd3a27222886ca8ed0bd7acc0c873c9fcf16f7289ebf96688eef6ad4bd51"}
Oct 30 22:05:05.068896 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.068842    5817 scope.go:122] "RemoveContainer" containerID="e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e"
Oct 30 22:05:05.078857 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.078839    5817 scope.go:122] "RemoveContainer" containerID="e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e"
Oct 30 22:05:05.079147 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:05:05.079129    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e\": container with ID starting with e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e not found: ID does not exist" containerID="e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e"
Oct 30 22:05:05.079217 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.079157    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e"} err="failed to get container status \"e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e\": rpc error: code = NotFound desc = could not find container \"e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e\": container with ID starting with e2fc9a086ad4bb56a979c0af43a6e0e063af8ad771b2aa530ac0953ef20e409e not found: ID does not exist"
Oct 30 22:05:05.225228 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.225199    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"]
Oct 30 22:05:05.230013 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.229989    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ed76b8-openshift-default-68765db7c-r4mb5"]
Oct 30 22:05:05.374650 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.374622    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"]
Oct 30 22:05:05.375128 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375106    5817 cpu_manager.go:414] "containerMap: removing container" podUID="122cf522-ccbf-409b-9956-baf8a6ff103a" containerName="maas-api"
Oct 30 22:05:05.375128 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375125    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="122cf522-ccbf-409b-9956-baf8a6ff103a" containerName="maas-api"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375139    5817 cpu_manager.go:414] "containerMap: removing container" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375144    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375154    5817 cpu_manager.go:414] "containerMap: removing container" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375159    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375168    5817 cpu_manager.go:414] "containerMap: removing container" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375172    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375179    5817 cpu_manager.go:414] "containerMap: removing container" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375184    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375191    5817 cpu_manager.go:414] "containerMap: removing container" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375196    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375202    5817 cpu_manager.go:414] "containerMap: removing container" podUID="efc0e828-5e19-4eed-87df-6ba8727d9c73" containerName="payload-processing"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375208    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efc0e828-5e19-4eed-87df-6ba8727d9c73" containerName="payload-processing"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375215    5817 cpu_manager.go:414] "containerMap: removing container" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375219    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375228    5817 cpu_manager.go:414] "containerMap: removing container" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375232    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375238    5817 cpu_manager.go:414] "containerMap: removing container" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375244    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375252    5817 cpu_manager.go:414] "containerMap: removing container" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375256    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375265    5817 cpu_manager.go:414] "containerMap: removing container" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375269    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy"
Oct 30 22:05:05.375266 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375277    5817 cpu_manager.go:414] "containerMap: removing container" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375282    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375296    5817 cpu_manager.go:414] "containerMap: removing container" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375300    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375306    5817 cpu_manager.go:414] "containerMap: removing container" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375310    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375319    5817 cpu_manager.go:414] "containerMap: removing container" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375323    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375329    5817 cpu_manager.go:414] "containerMap: removing container" podUID="c8e358dd-6be9-4c08-a230-f4c9a377bb3a" containerName="maas-api"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375333    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8e358dd-6be9-4c08-a230-f4c9a377bb3a" containerName="maas-api"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375341    5817 cpu_manager.go:414] "containerMap: removing container" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375360    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375372    5817 cpu_manager.go:414] "containerMap: removing container" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375376    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375384    5817 cpu_manager.go:414] "containerMap: removing container" podUID="df85e97b-dc4d-4b76-8977-0c721046960d" containerName="payload-pre-processing"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375389    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df85e97b-dc4d-4b76-8977-0c721046960d" containerName="payload-pre-processing"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375404    5817 cpu_manager.go:414] "containerMap: removing container" podUID="5930a3d6-c4e4-4518-bee4-de0e1340a990" containerName="maas-api"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375409    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5930a3d6-c4e4-4518-bee4-de0e1340a990" containerName="maas-api"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375416    5817 cpu_manager.go:414] "containerMap: removing container" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375420    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375429    5817 cpu_manager.go:414] "containerMap: removing container" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375433    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375439    5817 cpu_manager.go:414] "containerMap: removing container" podUID="70a9167f-e9f7-4108-876c-22c4784f2c94" containerName="payload-pre-processing"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375443    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70a9167f-e9f7-4108-876c-22c4784f2c94" containerName="payload-pre-processing"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375514    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375523    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375531    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375536    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b5efc58-2ff8-4a72-a352-925e7bbe3a24" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375542    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375547    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8e358dd-6be9-4c08-a230-f4c9a377bb3a" containerName="maas-api"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375554    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="c289aead-2be5-4573-9f7b-52be0b91b408" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375560    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="28a09d72-9c45-41d6-b755-d018dc2271cb" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375565    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="df85e97b-dc4d-4b76-8977-0c721046960d" containerName="payload-pre-processing"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375571    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375578    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="5930a3d6-c4e4-4518-bee4-de0e1340a990" containerName="maas-api"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375583    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb8a0fb2-6236-41c4-9fde-34d97a9a71d3" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375588    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="9680477d-1d4e-45b5-b96b-f20a0044fd3e" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375593    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="70a9167f-e9f7-4108-876c-22c4784f2c94" containerName="payload-pre-processing"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375600    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="122cf522-ccbf-409b-9956-baf8a6ff103a" containerName="maas-api"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375605    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="a18e6ea2-6992-427d-8c0f-6244c76052d6" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375610    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375616    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="57ecfea3-2e0a-4ead-b994-6bc7aca1aa52" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375621    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="313a4a3a-5ea1-4090-b8d6-40f181ffd3aa" containerName="main"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375625    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d54ac74-962b-4549-8c9b-a2c98f4d3251" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375631    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="efc0e828-5e19-4eed-87df-6ba8727d9c73" containerName="payload-processing"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375636    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="11adea8b-7c70-46c4-84cd-d3ed86fcd343" containerName="istio-proxy"
Oct 30 22:05:05.375975 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.375642    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy"
Oct 30 22:05:05.380318 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.380298    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.438251 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.438229    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"]
Oct 30 22:05:05.478992 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.478938    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:05:05.479289 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.479272    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c6npn\""
Oct 30 22:05:05.510799 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.510775    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.510890 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.510826    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.510944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.510910    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq5nv\" (UniqueName: \"kubernetes.io/projected/d49ca124-33d8-4457-a5b7-7025c36243b8-kube-api-access-bq5nv\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.611593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.611563    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.611593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.611595    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq5nv\" (UniqueName: \"kubernetes.io/projected/d49ca124-33d8-4457-a5b7-7025c36243b8-kube-api-access-bq5nv\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.611760 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.611688    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.612209 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.612182    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.613982 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.613961    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.670337 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.670310    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq5nv\" (UniqueName: \"kubernetes.io/projected/d49ca124-33d8-4457-a5b7-7025c36243b8-kube-api-access-bq5nv\") pod \"maas-api-revoke-sub-02b30ebe-clmw8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:05.698298 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:05.698269    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:06.064735 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:06.064713    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"]
Oct 30 22:05:06.067235 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:05:06.067208    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49ca124_33d8_4457_a5b7_7025c36243b8.slice/crio-3bd5d65cc4d77974d9b21230f24bdf46f995ec181c8263cb64e697d7a9fcffb7 WatchSource:0}: Error finding container 3bd5d65cc4d77974d9b21230f24bdf46f995ec181c8263cb64e697d7a9fcffb7: Status 404 returned error can't find the container with id 3bd5d65cc4d77974d9b21230f24bdf46f995ec181c8263cb64e697d7a9fcffb7
Oct 30 22:05:06.078185 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:06.078149    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8" event={"ID":"d49ca124-33d8-4457-a5b7-7025c36243b8","Type":"ContainerStarted","Data":"3bd5d65cc4d77974d9b21230f24bdf46f995ec181c8263cb64e697d7a9fcffb7"}
Oct 30 22:05:06.841875 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:06.841840    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20f96548-9ebe-4495-8aea-c3baf811f695" path="/var/lib/kubelet/pods/20f96548-9ebe-4495-8aea-c3baf811f695/volumes"
Oct 30 22:05:07.086257 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:07.086223    5817 generic.go:356] "Generic (PLEG): container finished" podID="d49ca124-33d8-4457-a5b7-7025c36243b8" containerID="ae00d1a8af0bfc0499e944716bf24ff54fa5c633a34c7ef4e4b9be174f4b1214" exitCode=0
Oct 30 22:05:07.086707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:07.086296    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8" event={"ID":"d49ca124-33d8-4457-a5b7-7025c36243b8","Type":"ContainerDied","Data":"ae00d1a8af0bfc0499e944716bf24ff54fa5c633a34c7ef4e4b9be174f4b1214"}
Oct 30 22:05:08.220774 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.220748    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:08.336046 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.336014    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-xks-ca\") pod \"d49ca124-33d8-4457-a5b7-7025c36243b8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") "
Oct 30 22:05:08.336218 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.336201    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d49ca124-33d8-4457-a5b7-7025c36243b8-kube-api-access-bq5nv\" (UniqueName: \"kubernetes.io/projected/d49ca124-33d8-4457-a5b7-7025c36243b8-kube-api-access-bq5nv\") pod \"d49ca124-33d8-4457-a5b7-7025c36243b8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") "
Oct 30 22:05:08.336281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.336266    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-openshift-ca\") pod \"d49ca124-33d8-4457-a5b7-7025c36243b8\" (UID: \"d49ca124-33d8-4457-a5b7-7025c36243b8\") "
Oct 30 22:05:08.336701 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.336670    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-openshift-ca" pod "d49ca124-33d8-4457-a5b7-7025c36243b8" (UID: "d49ca124-33d8-4457-a5b7-7025c36243b8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:05:08.338130 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.338105    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-xks-ca" pod "d49ca124-33d8-4457-a5b7-7025c36243b8" (UID: "d49ca124-33d8-4457-a5b7-7025c36243b8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:05:08.338236 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.338151    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49ca124-33d8-4457-a5b7-7025c36243b8-kube-api-access-bq5nv" pod "d49ca124-33d8-4457-a5b7-7025c36243b8" (UID: "d49ca124-33d8-4457-a5b7-7025c36243b8"). InnerVolumeSpecName "kube-api-access-bq5nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:05:08.437933 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.437905    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq5nv\" (UniqueName: \"kubernetes.io/projected/d49ca124-33d8-4457-a5b7-7025c36243b8-kube-api-access-bq5nv\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:08.437933 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.437934    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-openshift-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:08.438101 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:08.437949    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d49ca124-33d8-4457-a5b7-7025c36243b8-maas-api-xks-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:05:09.097159 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:09.097114    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8" event={"ID":"d49ca124-33d8-4457-a5b7-7025c36243b8","Type":"ContainerDied","Data":"3bd5d65cc4d77974d9b21230f24bdf46f995ec181c8263cb64e697d7a9fcffb7"}
Oct 30 22:05:09.097329 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:09.097175    5817 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bd5d65cc4d77974d9b21230f24bdf46f995ec181c8263cb64e697d7a9fcffb7"
Oct 30 22:05:09.097329 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:09.097130    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"
Oct 30 22:05:16.336832 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:05:16.336794    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:05:38.842097 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842027    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-744f599cc5-jwx27"]
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842427    5817 cpu_manager.go:414] "containerMap: removing container" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842438    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd7217ba-460e-45c6-804b-70ebd50c3d38" containerName="istio-proxy"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842456    5817 cpu_manager.go:414] "containerMap: removing container" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842462    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842479    5817 cpu_manager.go:414] "containerMap: removing container" podUID="d49ca124-33d8-4457-a5b7-7025c36243b8" containerName="revoke-keys"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842485    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d49ca124-33d8-4457-a5b7-7025c36243b8" containerName="revoke-keys"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842557    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="8972290a-50d8-4ad3-8998-88a670362dae" containerName="istio-proxy"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842566    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b8839ba-a89c-43b8-a9eb-c4c96b232b2d" containerName="istio-proxy"
Oct 30 22:05:38.842570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842574    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff45733c-4a30-43c1-b509-18792c82ec04" containerName="istio-proxy"
Oct 30 22:05:38.842855 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.842583    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="d49ca124-33d8-4457-a5b7-7025c36243b8" containerName="revoke-keys"
Oct 30 22:05:38.847218 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.847196    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:38.897037 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.897013    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:05:38.943721 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.943693    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5nx5r\""
Oct 30 22:05:38.943902 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.943885    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:05:38.943983 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.943914    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:05:38.944115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.944094    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnqbz\" (UniqueName: \"kubernetes.io/projected/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-kube-api-access-tnqbz\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:38.944193 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.944133    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-webhook-cert\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:38.944289 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.944266    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-metrics-certs\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:38.945778 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:38.945755    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-744f599cc5-jwx27"]
Oct 30 22:05:39.045550 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.045517    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnqbz\" (UniqueName: \"kubernetes.io/projected/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-kube-api-access-tnqbz\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:39.045708 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.045571    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-webhook-cert\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:39.045708 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.045615    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-metrics-certs\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:39.048044 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.048020    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-webhook-cert\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:39.048167 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.048128    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-metrics-certs\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:39.103704 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.103642    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnqbz\" (UniqueName: \"kubernetes.io/projected/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-kube-api-access-tnqbz\") pod \"maas-controller-744f599cc5-jwx27\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") " pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:39.160758 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.160722    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:39.534563 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.534537    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-744f599cc5-jwx27"]
Oct 30 22:05:39.537053 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:05:39.537013    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf60860d_0ace_4f00_a3fd_e5f9f056ebdd.slice/crio-d2c32c14e760614f68f89d4b67b252f1375cf126ffe9b88d89ce380ab26ab8dd WatchSource:0}: Error finding container d2c32c14e760614f68f89d4b67b252f1375cf126ffe9b88d89ce380ab26ab8dd: Status 404 returned error can't find the container with id d2c32c14e760614f68f89d4b67b252f1375cf126ffe9b88d89ce380ab26ab8dd
Oct 30 22:05:39.538637 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:39.538615    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:05:40.247659 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:40.247617    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-744f599cc5-jwx27" event={"ID":"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd","Type":"ContainerStarted","Data":"d2c32c14e760614f68f89d4b67b252f1375cf126ffe9b88d89ce380ab26ab8dd"}
Oct 30 22:05:43.299062 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:43.299020    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-744f599cc5-jwx27" event={"ID":"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd","Type":"ContainerStarted","Data":"62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e"}
Oct 30 22:05:43.309447 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:43.309414    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:46.336843 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:05:46.336802    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:05:54.306913 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:54.306879    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:05:54.423368 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:05:54.423299    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-744f599cc5-jwx27" podStartSLOduration=13.702787764 podStartE2EDuration="16.423286133s" totalImagesPullingTime="2.720498369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:43.429082305 +0000 UTC m=+3078.716804854" watchObservedRunningTime="2026-10-30 22:05:54.423286133 +0000 UTC m=+3089.711008693"
Oct 30 22:06:14.718151 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:14.718118    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-744f599cc5-jwx27"]
Oct 30 22:06:14.718658 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:14.718308    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-744f599cc5-jwx27" podUID="cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" containerName="manager" containerID="cri-o://62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e" gracePeriod=30
Oct 30 22:06:14.977187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:14.977132    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:06:15.144776 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.144739    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-webhook-cert\") pod \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") "
Oct 30 22:06:15.144968 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.144831    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-metrics-certs\") pod \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") "
Oct 30 22:06:15.144968 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.144869    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-kube-api-access-tnqbz\" (UniqueName: \"kubernetes.io/projected/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-kube-api-access-tnqbz\") pod \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\" (UID: \"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd\") "
Oct 30 22:06:15.146982 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.146945    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-metrics-certs" pod "cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" (UID: "cf60860d-0ace-4f00-a3fd-e5f9f056ebdd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:06:15.147126 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.147069    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-kube-api-access-tnqbz" pod "cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" (UID: "cf60860d-0ace-4f00-a3fd-e5f9f056ebdd"). InnerVolumeSpecName "kube-api-access-tnqbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:15.147213 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.147188    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-webhook-cert" pod "cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" (UID: "cf60860d-0ace-4f00-a3fd-e5f9f056ebdd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:06:15.246517 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.246427    5817 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-webhook-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:06:15.246517 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.246462    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:06:15.246517 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.246476    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnqbz\" (UniqueName: \"kubernetes.io/projected/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd-kube-api-access-tnqbz\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:06:15.451588 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.451556    5817 generic.go:356] "Generic (PLEG): container finished" podID="cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" containerID="62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e" exitCode=0
Oct 30 22:06:15.451761 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.451602    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-744f599cc5-jwx27"
Oct 30 22:06:15.451761 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.451642    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-744f599cc5-jwx27" event={"ID":"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd","Type":"ContainerDied","Data":"62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e"}
Oct 30 22:06:15.451761 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.451694    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-744f599cc5-jwx27" event={"ID":"cf60860d-0ace-4f00-a3fd-e5f9f056ebdd","Type":"ContainerDied","Data":"d2c32c14e760614f68f89d4b67b252f1375cf126ffe9b88d89ce380ab26ab8dd"}
Oct 30 22:06:15.451761 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.451717    5817 scope.go:122] "RemoveContainer" containerID="62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e"
Oct 30 22:06:15.461011 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.460994    5817 scope.go:122] "RemoveContainer" containerID="62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e"
Oct 30 22:06:15.461249 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:06:15.461234    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e\": container with ID starting with 62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e not found: ID does not exist" containerID="62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e"
Oct 30 22:06:15.461295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.461257    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e"} err="failed to get container status \"62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e\": rpc error: code = NotFound desc = could not find container \"62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e\": container with ID starting with 62d57cee4e4df152a9478a429c2896dc83d84f813cd57526140c8f15b9bb809e not found: ID does not exist"
Oct 30 22:06:15.607022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.606960    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-744f599cc5-jwx27"]
Oct 30 22:06:15.653932 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:15.653902    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-744f599cc5-jwx27"]
Oct 30 22:06:16.338432 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:06:16.338399    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:06:16.842294 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:16.842261    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" path="/var/lib/kubelet/pods/cf60860d-0ace-4f00-a3fd-e5f9f056ebdd/volumes"
Oct 30 22:06:17.136705 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.136632    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-744f599cc5-9q2r7"]
Oct 30 22:06:17.137107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.137092    5817 cpu_manager.go:414] "containerMap: removing container" podUID="cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" containerName="manager"
Oct 30 22:06:17.137153 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.137109    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" containerName="manager"
Oct 30 22:06:17.137197 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.137187    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf60860d-0ace-4f00-a3fd-e5f9f056ebdd" containerName="manager"
Oct 30 22:06:17.142094 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.142074    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.239140 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.239087    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:06:17.239140 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.239122    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:06:17.239399 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.239161    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:06:17.239399 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.239298    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5nx5r\""
Oct 30 22:06:17.242958 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.242930    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-744f599cc5-9q2r7"]
Oct 30 22:06:17.267290 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.267262    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6767a5f0-120f-4614-9fb0-4b251243b773-webhook-cert\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.267458 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.267326    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6767a5f0-120f-4614-9fb0-4b251243b773-metrics-certs\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.267510 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.267491    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg7v6\" (UniqueName: \"kubernetes.io/projected/6767a5f0-120f-4614-9fb0-4b251243b773-kube-api-access-wg7v6\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.368825 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.368744    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6767a5f0-120f-4614-9fb0-4b251243b773-webhook-cert\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.368825 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.368802    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6767a5f0-120f-4614-9fb0-4b251243b773-metrics-certs\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.368825 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.368832    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg7v6\" (UniqueName: \"kubernetes.io/projected/6767a5f0-120f-4614-9fb0-4b251243b773-kube-api-access-wg7v6\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.371394 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.371370    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6767a5f0-120f-4614-9fb0-4b251243b773-metrics-certs\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.371514 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.371410    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6767a5f0-120f-4614-9fb0-4b251243b773-webhook-cert\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.436680 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.436612    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg7v6\" (UniqueName: \"kubernetes.io/projected/6767a5f0-120f-4614-9fb0-4b251243b773-kube-api-access-wg7v6\") pod \"maas-controller-744f599cc5-9q2r7\" (UID: \"6767a5f0-120f-4614-9fb0-4b251243b773\") " pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.455515 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.455487    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:17.823729 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:17.823690    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-744f599cc5-9q2r7"]
Oct 30 22:06:17.826318 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:06:17.826287    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6767a5f0_120f_4614_9fb0_4b251243b773.slice/crio-ef3ba14f69141f71bf6c7dc54ac5fca4d26ab4014375d8da92f589a2081d0f73 WatchSource:0}: Error finding container ef3ba14f69141f71bf6c7dc54ac5fca4d26ab4014375d8da92f589a2081d0f73: Status 404 returned error can't find the container with id ef3ba14f69141f71bf6c7dc54ac5fca4d26ab4014375d8da92f589a2081d0f73
Oct 30 22:06:18.471668 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:18.471634    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-744f599cc5-9q2r7" event={"ID":"6767a5f0-120f-4614-9fb0-4b251243b773","Type":"ContainerStarted","Data":"ef3ba14f69141f71bf6c7dc54ac5fca4d26ab4014375d8da92f589a2081d0f73"}
Oct 30 22:06:19.479504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:19.479466    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-744f599cc5-9q2r7" event={"ID":"6767a5f0-120f-4614-9fb0-4b251243b773","Type":"ContainerStarted","Data":"9a746599984092a06109339df5cf16a60c920fac5e664f664da80910fb9223c6"}
Oct 30 22:06:19.479905 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:19.479566    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:19.592902 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:19.592856    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-744f599cc5-9q2r7" podStartSLOduration=1.671508152 podStartE2EDuration="2.592842368s" totalImagesPullingTime="921.334216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:19.591140065 +0000 UTC m=+3114.878862612" watchObservedRunningTime="2026-10-30 22:06:19.592842368 +0000 UTC m=+3114.880564913"
Oct 30 22:06:30.486689 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:06:30.486659    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-744f599cc5-9q2r7"
Oct 30 22:06:46.337468 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:06:46.337426    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:07:16.337309 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:07:16.337218    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:07:46.338025 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:07:46.337990    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:08:16.456796 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:08:16.456753    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:08:17.352490 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:08:17.352447    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:08:17.352814 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:08:17.352778    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:08:46.337812 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:08:46.337771    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:09:16.338893 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:09:16.338854    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:09:46.337610 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:09:46.337572    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:10:09.053646 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:10:09.053547    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"]
Oct 30 22:10:09.101370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:10:09.100273    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-02b30ebe-clmw8"]
Oct 30 22:10:10.882141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:10:10.881949    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d49ca124-33d8-4457-a5b7-7025c36243b8" path="/var/lib/kubelet/pods/d49ca124-33d8-4457-a5b7-7025c36243b8/volumes"
Oct 30 22:10:16.337159 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:10:16.337124    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:10:46.336671 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:10:46.336637    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:11:16.443669 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:11:16.443605    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:11:17.363374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:11:17.362443    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:11:17.363374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:11:17.362518    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:11:27.638612 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:11:27.638577    5817 scope.go:122] "RemoveContainer" containerID="ae00d1a8af0bfc0499e944716bf24ff54fa5c633a34c7ef4e4b9be174f4b1214"
Oct 30 22:11:46.337000 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:11:46.336954    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:12:16.337071 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:12:16.337034    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:12:46.336723 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:12:46.336681    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:13:16.337209 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:13:16.337173    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:13:19.635105 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.635075    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"]
Oct 30 22:13:19.639007 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.638982    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725419 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725387    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-socket\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725609 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725429    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istiod-ca-cert\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725609 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725464    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-certs\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725751 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725616    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8htl\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-kube-api-access-m8htl\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725751 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725675    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-token\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725751 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725723    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-podinfo\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725854 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725808    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-credential-socket\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725907 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725867    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-data\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.725967 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.725919    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-envoy\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.750246 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.750221    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-788e42-openshift-default-dockercfg-g7ht5\""
Oct 30 22:13:19.806379 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.806332    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"]
Oct 30 22:13:19.826641 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826612    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-socket\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.826797 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826648    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istiod-ca-cert\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.826797 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826686    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-certs\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.826797 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826742    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8htl\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-kube-api-access-m8htl\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.826797 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826783    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-token\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827002 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826817    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-podinfo\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827002 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826853    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-credential-socket\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827002 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826885    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-data\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827002 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.826908    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-envoy\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827203 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.827036    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-socket\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827362 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.827321    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-credential-socket\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827485 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.827325    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-data\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827485 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.827413    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-certs\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.827564 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.827513    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istiod-ca-cert\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.829251 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.829228    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-envoy\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.829390 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.829336    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-podinfo\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.977605 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.977572    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-token\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:19.977775 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:19.977700    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8htl\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-kube-api-access-m8htl\") pod \"e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") " pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:20.251395 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:20.251294    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:20.672136 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:20.671971    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"]
Oct 30 22:13:20.674687 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:13:20.674642    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc698b74_0a5d_4894_ad5a_4d2be1816a91.slice/crio-2f954fc17c01015635a73710a45dff6e9ca9d1296a7170253e2d1db0c186d9d4 WatchSource:0}: Error finding container 2f954fc17c01015635a73710a45dff6e9ca9d1296a7170253e2d1db0c186d9d4: Status 404 returned error can't find the container with id 2f954fc17c01015635a73710a45dff6e9ca9d1296a7170253e2d1db0c186d9d4
Oct 30 22:13:20.676942 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:20.676907    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:20.677040 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:20.676987    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:20.677040 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:20.677033    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:21.017963 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:21.017916    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" event={"ID":"fc698b74-0a5d-4894-ad5a-4d2be1816a91","Type":"ContainerStarted","Data":"a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584"}
Oct 30 22:13:21.018172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:21.017987    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" event={"ID":"fc698b74-0a5d-4894-ad5a-4d2be1816a91","Type":"ContainerStarted","Data":"2f954fc17c01015635a73710a45dff6e9ca9d1296a7170253e2d1db0c186d9d4"}
Oct 30 22:13:21.227815 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:21.227752    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" podStartSLOduration=2.227736703 podStartE2EDuration="2.227736703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:21.226885645 +0000 UTC m=+3536.514608194" watchObservedRunningTime="2026-10-30 22:13:21.227736703 +0000 UTC m=+3536.515459249"
Oct 30 22:13:21.251886 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:21.251855    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:21.257076 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:21.257050    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:22.023285 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:22.023252    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:22.024213 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:22.024191    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:27.946445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:27.946395    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"]
Oct 30 22:13:27.950192 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:27.950169    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:27.999784 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:27.999759    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-788e42\""
Oct 30 22:13:28.014886 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.014855    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:28.015035 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.014899    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr472\" (UniqueName: \"kubernetes.io/projected/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-kube-api-access-gr472\") pod \"payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:28.048037 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.048007    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"]
Oct 30 22:13:28.112785 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.112748    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"]
Oct 30 22:13:28.116408 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.116374    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:28.116528 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.116439    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr472\" (UniqueName: \"kubernetes.io/projected/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-kube-api-access-gr472\") pod \"payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:28.116775 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.116758    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.117250 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.117227    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:28.170680 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.170650    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"]
Oct 30 22:13:28.215110 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.215081    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-89e3164f-dockercfg-9vz6j\""
Oct 30 22:13:28.217200 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.217171    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khg4t\" (UniqueName: \"kubernetes.io/projected/955a3c91-6234-4799-8656-0c2596f708de-kube-api-access-khg4t\") pod \"payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.217318 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.217228    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955a3c91-6234-4799-8656-0c2596f708de-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.217617 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.217598    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr472\" (UniqueName: \"kubernetes.io/projected/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-kube-api-access-gr472\") pod \"payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:28.263162 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.263138    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:28.321942 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.318291    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khg4t\" (UniqueName: \"kubernetes.io/projected/955a3c91-6234-4799-8656-0c2596f708de-kube-api-access-khg4t\") pod \"payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.321942 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.318396    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955a3c91-6234-4799-8656-0c2596f708de-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.321942 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.319071    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955a3c91-6234-4799-8656-0c2596f708de-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.377193 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.377164    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khg4t\" (UniqueName: \"kubernetes.io/projected/955a3c91-6234-4799-8656-0c2596f708de-kube-api-access-khg4t\") pod \"payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.430245 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.430214    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:28.630457 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.630427    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"]
Oct 30 22:13:28.632965 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:13:28.632936    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f31ff5_dddc_4f1c_b1d3_c57bcd8a07c3.slice/crio-04ee3088705e3c5e9815b6121215c464e484d8077156fc33db3bab14dcbb754f WatchSource:0}: Error finding container 04ee3088705e3c5e9815b6121215c464e484d8077156fc33db3bab14dcbb754f: Status 404 returned error can't find the container with id 04ee3088705e3c5e9815b6121215c464e484d8077156fc33db3bab14dcbb754f
Oct 30 22:13:28.799114 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:28.799091    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"]
Oct 30 22:13:28.801467 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:13:28.801438    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955a3c91_6234_4799_8656_0c2596f708de.slice/crio-ef12a5a41a6e1d014981fd77f7bd416e3c4bb3d49264c54ecb7c530c7ca37faf WatchSource:0}: Error finding container ef12a5a41a6e1d014981fd77f7bd416e3c4bb3d49264c54ecb7c530c7ca37faf: Status 404 returned error can't find the container with id ef12a5a41a6e1d014981fd77f7bd416e3c4bb3d49264c54ecb7c530c7ca37faf
Oct 30 22:13:29.058743 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.058655    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" event={"ID":"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3","Type":"ContainerStarted","Data":"4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3"}
Oct 30 22:13:29.058743 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.058718    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" event={"ID":"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3","Type":"ContainerStarted","Data":"04ee3088705e3c5e9815b6121215c464e484d8077156fc33db3bab14dcbb754f"}
Oct 30 22:13:29.059241 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.058834    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:29.060234 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.060213    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" event={"ID":"955a3c91-6234-4799-8656-0c2596f708de","Type":"ContainerStarted","Data":"ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba"}
Oct 30 22:13:29.060330 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.060252    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" event={"ID":"955a3c91-6234-4799-8656-0c2596f708de","Type":"ContainerStarted","Data":"ef12a5a41a6e1d014981fd77f7bd416e3c4bb3d49264c54ecb7c530c7ca37faf"}
Oct 30 22:13:29.060402 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.060371    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:29.169632 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.169583    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" podStartSLOduration=2.169569824 podStartE2EDuration="2.169569824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:29.167979903 +0000 UTC m=+3544.455702450" watchObservedRunningTime="2026-10-30 22:13:29.169569824 +0000 UTC m=+3544.457292370"
Oct 30 22:13:29.280945 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:29.280866    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" podStartSLOduration=1.280846794 podStartE2EDuration="1.280846794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:29.278601163 +0000 UTC m=+3544.566323712" watchObservedRunningTime="2026-10-30 22:13:29.280846794 +0000 UTC m=+3544.568569343"
Oct 30 22:13:40.072023 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:40.071980    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:13:40.072546 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:40.072052    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:13:46.339661 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:13:46.339606    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:13:49.166413 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:49.166377    5817 generic.go:356] "Generic (PLEG): container finished" podID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerID="a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584" exitCode=0
Oct 30 22:13:49.166846 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:49.166426    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" event={"ID":"fc698b74-0a5d-4894-ad5a-4d2be1816a91","Type":"ContainerDied","Data":"a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584"}
Oct 30 22:13:49.166846 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:49.166778    5817 scope.go:122] "RemoveContainer" containerID="a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584"
Oct 30 22:13:49.167939 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:49.167912    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:49.167990 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:49.167980    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:49.168025 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:49.168011    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:13:50.173653 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:50.173618    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" event={"ID":"fc698b74-0a5d-4894-ad5a-4d2be1816a91","Type":"ContainerStarted","Data":"3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969"}
Oct 30 22:13:50.251806 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:50.251768    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:50.253106 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:50.253079    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:13:50.253226 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:50.253132    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:13:51.252185 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:51.252149    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:13:51.252624 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:51.252205    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:13:52.024918 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:52.024878    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:52.252694 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:52.252655    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:13:52.253103 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:52.252710    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:13:53.252026 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:53.251984    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:13:53.252241 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:53.252043    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:13:54.252504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:54.252459    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 22:13:54.253005 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:54.252533    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 22:13:55.256258 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:55.256218    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:13:55.257305 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:13:55.257279    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:14:16.339265 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:14:16.339221    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:14:46.337895 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:14:46.337857    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:15:00.192291 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.192257    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"]
Oct 30 22:15:00.196264 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.196238    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.251644 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.251615    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-c6npn\""
Oct 30 22:15:00.259149 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.259111    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfs7d\" (UniqueName: \"kubernetes.io/projected/cb4099a6-8809-4b25-a60e-f851c6f2bff1-kube-api-access-cfs7d\") pod \"maas-api-key-cleanup-29889975-t2qhx\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.259309 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.259155    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4099a6-8809-4b25-a60e-f851c6f2bff1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-t2qhx\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.300556 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.300515    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"]
Oct 30 22:15:00.360961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.360912    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfs7d\" (UniqueName: \"kubernetes.io/projected/cb4099a6-8809-4b25-a60e-f851c6f2bff1-kube-api-access-cfs7d\") pod \"maas-api-key-cleanup-29889975-t2qhx\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.360961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.360961    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4099a6-8809-4b25-a60e-f851c6f2bff1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-t2qhx\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.361722 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.361701    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4099a6-8809-4b25-a60e-f851c6f2bff1-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889975-t2qhx\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.421996 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.421959    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfs7d\" (UniqueName: \"kubernetes.io/projected/cb4099a6-8809-4b25-a60e-f851c6f2bff1-kube-api-access-cfs7d\") pod \"maas-api-key-cleanup-29889975-t2qhx\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.510339 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.510305    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:00.895064 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:00.895036    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"]
Oct 30 22:15:00.897334 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:15:00.897298    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4099a6_8809_4b25_a60e_f851c6f2bff1.slice/crio-d8345725ebcd33806d80cee31d5672d97a991d04cbfe8fcd167b03a3f30f3873 WatchSource:0}: Error finding container d8345725ebcd33806d80cee31d5672d97a991d04cbfe8fcd167b03a3f30f3873: Status 404 returned error can't find the container with id d8345725ebcd33806d80cee31d5672d97a991d04cbfe8fcd167b03a3f30f3873
Oct 30 22:15:01.590568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:01.590527    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" event={"ID":"cb4099a6-8809-4b25-a60e-f851c6f2bff1","Type":"ContainerStarted","Data":"3fa35238a10f9a69f92dd97d01c19024ec2dbb269a9a5e5c25414deef9fb627f"}
Oct 30 22:15:01.591059 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:01.590592    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" event={"ID":"cb4099a6-8809-4b25-a60e-f851c6f2bff1","Type":"ContainerStarted","Data":"d8345725ebcd33806d80cee31d5672d97a991d04cbfe8fcd167b03a3f30f3873"}
Oct 30 22:15:01.710681 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:01.710633    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" podStartSLOduration=1.710619537 podStartE2EDuration="1.710619537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:01.709722921 +0000 UTC m=+3636.997445466" watchObservedRunningTime="2026-10-30 22:15:01.710619537 +0000 UTC m=+3636.998342083"
Oct 30 22:15:01.810570 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:01.810534    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"]
Oct 30 22:15:01.902466 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:01.902382    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"]
Oct 30 22:15:02.597115 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:02.597078    5817 generic.go:356] "Generic (PLEG): container finished" podID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerID="3fa35238a10f9a69f92dd97d01c19024ec2dbb269a9a5e5c25414deef9fb627f" exitCode=60
Oct 30 22:15:02.597548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:02.597127    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" event={"ID":"cb4099a6-8809-4b25-a60e-f851c6f2bff1","Type":"ContainerDied","Data":"3fa35238a10f9a69f92dd97d01c19024ec2dbb269a9a5e5c25414deef9fb627f"}
Oct 30 22:15:02.597548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:02.597401    5817 scope.go:122] "RemoveContainer" containerID="3fa35238a10f9a69f92dd97d01c19024ec2dbb269a9a5e5c25414deef9fb627f"
Oct 30 22:15:03.604616 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:03.604517    5817 generic.go:356] "Generic (PLEG): container finished" podID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerID="4ac035b7541f3b82887107eef7b846016d96af7790e55073ba0225f01229ce74" exitCode=60
Oct 30 22:15:03.604616 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:03.604552    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" event={"ID":"cb4099a6-8809-4b25-a60e-f851c6f2bff1","Type":"ContainerDied","Data":"4ac035b7541f3b82887107eef7b846016d96af7790e55073ba0225f01229ce74"}
Oct 30 22:15:03.605142 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:03.604626    5817 scope.go:122] "RemoveContainer" containerID="3fa35238a10f9a69f92dd97d01c19024ec2dbb269a9a5e5c25414deef9fb627f"
Oct 30 22:15:03.605142 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:03.604894    5817 scope.go:122] "RemoveContainer" containerID="4ac035b7541f3b82887107eef7b846016d96af7790e55073ba0225f01229ce74"
Oct 30 22:15:03.605246 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:03.605176    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-t2qhx_odh-ai-gateway-infra(cb4099a6-8809-4b25-a60e-f851c6f2bff1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1"
Oct 30 22:15:04.611574 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:04.611547    5817 scope.go:122] "RemoveContainer" containerID="4ac035b7541f3b82887107eef7b846016d96af7790e55073ba0225f01229ce74"
Oct 30 22:15:04.611949 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:04.611769    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-t2qhx_odh-ai-gateway-infra(cb4099a6-8809-4b25-a60e-f851c6f2bff1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1"
Oct 30 22:15:06.811891 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:06.811842    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" podUID="955a3c91-6234-4799-8656-0c2596f708de" containerName="payload-processing" containerID="cri-o://ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba" gracePeriod=25
Oct 30 22:15:06.902659 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:06.902619    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" podUID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" containerName="payload-pre-processing" containerID="cri-o://4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3" gracePeriod=25
Oct 30 22:15:10.071513 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:10.071469    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" podUID="955a3c91-6234-4799-8656-0c2596f708de" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:10.071909 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:10.071662    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" podUID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:14.192705 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:14.192662    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"]
Oct 30 22:15:14.193231 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:14.192898    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy" containerID="cri-o://3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969" gracePeriod=30
Oct 30 22:15:14.839757 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:14.839723    5817 scope.go:122] "RemoveContainer" containerID="4ac035b7541f3b82887107eef7b846016d96af7790e55073ba0225f01229ce74"
Oct 30 22:15:15.669321 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:15.669288    5817 generic.go:356] "Generic (PLEG): container finished" podID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerID="40e70a53000982a0b287a8cbe46f2da4f45f292598a049eec39c5b6b777ee8d7" exitCode=60
Oct 30 22:15:15.669829 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:15.669377    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" event={"ID":"cb4099a6-8809-4b25-a60e-f851c6f2bff1","Type":"ContainerDied","Data":"40e70a53000982a0b287a8cbe46f2da4f45f292598a049eec39c5b6b777ee8d7"}
Oct 30 22:15:15.669829 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:15.669442    5817 scope.go:122] "RemoveContainer" containerID="4ac035b7541f3b82887107eef7b846016d96af7790e55073ba0225f01229ce74"
Oct 30 22:15:15.669963 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:15.669843    5817 scope.go:122] "RemoveContainer" containerID="40e70a53000982a0b287a8cbe46f2da4f45f292598a049eec39c5b6b777ee8d7"
Oct 30 22:15:15.670144 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:15.670120    5817 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889975-t2qhx_odh-ai-gateway-infra(cb4099a6-8809-4b25-a60e-f851c6f2bff1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1"
Oct 30 22:15:16.338399 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:16.338359    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:15:16.793704 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:16.793672    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"]
Oct 30 22:15:16.943883 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:16.943855    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:17.046688 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.046620    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb4099a6-8809-4b25-a60e-f851c6f2bff1-kube-api-access-cfs7d\" (UniqueName: \"kubernetes.io/projected/cb4099a6-8809-4b25-a60e-f851c6f2bff1-kube-api-access-cfs7d\") pod \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") "
Oct 30 22:15:17.046688 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.046652    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb4099a6-8809-4b25-a60e-f851c6f2bff1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4099a6-8809-4b25-a60e-f851c6f2bff1-maas-api-service-ca\") pod \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\" (UID: \"cb4099a6-8809-4b25-a60e-f851c6f2bff1\") "
Oct 30 22:15:17.047083 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.047061    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4099a6-8809-4b25-a60e-f851c6f2bff1-maas-api-service-ca" pod "cb4099a6-8809-4b25-a60e-f851c6f2bff1" (UID: "cb4099a6-8809-4b25-a60e-f851c6f2bff1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:17.047156 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.047141    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4099a6-8809-4b25-a60e-f851c6f2bff1-maas-api-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.048740 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.048714    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4099a6-8809-4b25-a60e-f851c6f2bff1-kube-api-access-cfs7d" pod "cb4099a6-8809-4b25-a60e-f851c6f2bff1" (UID: "cb4099a6-8809-4b25-a60e-f851c6f2bff1"). InnerVolumeSpecName "kube-api-access-cfs7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:17.147757 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.147725    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfs7d\" (UniqueName: \"kubernetes.io/projected/cb4099a6-8809-4b25-a60e-f851c6f2bff1-kube-api-access-cfs7d\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:17.682323 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.682287    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"
Oct 30 22:15:17.682532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.682320    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx" event={"ID":"cb4099a6-8809-4b25-a60e-f851c6f2bff1","Type":"ContainerDied","Data":"d8345725ebcd33806d80cee31d5672d97a991d04cbfe8fcd167b03a3f30f3873"}
Oct 30 22:15:17.682532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.682388    5817 scope.go:122] "RemoveContainer" containerID="40e70a53000982a0b287a8cbe46f2da4f45f292598a049eec39c5b6b777ee8d7"
Oct 30 22:15:17.838501 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.838465    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"]
Oct 30 22:15:17.841866 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:17.841841    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889975-t2qhx"]
Oct 30 22:15:18.842511 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:18.842475    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" path="/var/lib/kubelet/pods/cb4099a6-8809-4b25-a60e-f851c6f2bff1/volumes"
Oct 30 22:15:19.448681 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.448660    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:15:19.572905 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.572830    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-podinfo\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.572905 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.572903    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-kube-api-access-m8htl\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-kube-api-access-m8htl\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.572937    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-credential-socket\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.572992    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-socket\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573040    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-data\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573083    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-certs\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573122    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-token\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-token\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573441 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573171    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-envoy\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573441 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573242    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istiod-ca-cert\") pod \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\" (UID: \"fc698b74-0a5d-4894-ad5a-4d2be1816a91\") "
Oct 30 22:15:19.573441 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573305    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-socket" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:19.573441 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573304    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-credential-socket" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:19.573441 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573336    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-data" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:19.573707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573436    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-certs" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:19.573762 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573737    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.573805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573761    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.573805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573779    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.573882 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573815    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.573936 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.573873    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istiod-ca-cert" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:19.575399 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.575375    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-envoy" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:19.575524 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.575399    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-kube-api-access-m8htl" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "kube-api-access-m8htl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:19.575576 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.575556    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-podinfo" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:19.575703 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.575683    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-token" pod "fc698b74-0a5d-4894-ad5a-4d2be1816a91" (UID: "fc698b74-0a5d-4894-ad5a-4d2be1816a91"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:19.674556 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.674523    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.674556 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.674551    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.674556 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.674560    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.674780 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.674569    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc698b74-0a5d-4894-ad5a-4d2be1816a91-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.674780 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.674580    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8htl\" (UniqueName: \"kubernetes.io/projected/fc698b74-0a5d-4894-ad5a-4d2be1816a91-kube-api-access-m8htl\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:19.695473 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.695432    5817 generic.go:356] "Generic (PLEG): container finished" podID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerID="3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969" exitCode=0
Oct 30 22:15:19.695610 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.695514    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"
Oct 30 22:15:19.695610 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.695517    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" event={"ID":"fc698b74-0a5d-4894-ad5a-4d2be1816a91","Type":"ContainerDied","Data":"3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969"}
Oct 30 22:15:19.695610 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.695595    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8" event={"ID":"fc698b74-0a5d-4894-ad5a-4d2be1816a91","Type":"ContainerDied","Data":"2f954fc17c01015635a73710a45dff6e9ca9d1296a7170253e2d1db0c186d9d4"}
Oct 30 22:15:19.695781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.695618    5817 scope.go:122] "RemoveContainer" containerID="3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969"
Oct 30 22:15:19.705555 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.705539    5817 scope.go:122] "RemoveContainer" containerID="a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584"
Oct 30 22:15:19.713397 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.713378    5817 scope.go:122] "RemoveContainer" containerID="3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969"
Oct 30 22:15:19.713632 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:19.713616    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969\": container with ID starting with 3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969 not found: ID does not exist" containerID="3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969"
Oct 30 22:15:19.713696 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.713642    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969"} err="failed to get container status \"3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969\": rpc error: code = NotFound desc = could not find container \"3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969\": container with ID starting with 3bd182f468bd10ba0cdd33297dbe67a7e1768647757a4fa32cd8a64fe6f9a969 not found: ID does not exist"
Oct 30 22:15:19.713696 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.713666    5817 scope.go:122] "RemoveContainer" containerID="a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584"
Oct 30 22:15:19.713906 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:19.713888    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584\": container with ID starting with a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584 not found: ID does not exist" containerID="a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584"
Oct 30 22:15:19.713955 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.713910    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584"} err="failed to get container status \"a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584\": rpc error: code = NotFound desc = could not find container \"a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584\": container with ID starting with a184d95c4dbf16fc1d4798a01a11db5d89fcaace41436217ed70121095e58584 not found: ID does not exist"
Oct 30 22:15:19.852197 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.852125    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"]
Oct 30 22:15:19.899077 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:19.899049    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-788e42-openshift-default-66bddcc5ccl6js8"]
Oct 30 22:15:20.070440 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:20.070411    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" podUID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:20.070603 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:20.070432    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" podUID="955a3c91-6234-4799-8656-0c2596f708de" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:20.842215 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:20.842183    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" path="/var/lib/kubelet/pods/fc698b74-0a5d-4894-ad5a-4d2be1816a91/volumes"
Oct 30 22:15:27.162718 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.162692    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:15:27.184289 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.184266    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:15:27.248388 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.248356    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/955a3c91-6234-4799-8656-0c2596f708de-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955a3c91-6234-4799-8656-0c2596f708de-plugins-config-volume\") pod \"955a3c91-6234-4799-8656-0c2596f708de\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") "
Oct 30 22:15:27.248566 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.248477    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/955a3c91-6234-4799-8656-0c2596f708de-kube-api-access-khg4t\" (UniqueName: \"kubernetes.io/projected/955a3c91-6234-4799-8656-0c2596f708de-kube-api-access-khg4t\") pod \"955a3c91-6234-4799-8656-0c2596f708de\" (UID: \"955a3c91-6234-4799-8656-0c2596f708de\") "
Oct 30 22:15:27.248566 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.248523    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-plugins-config-volume\") pod \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") "
Oct 30 22:15:27.248566 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.248540    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-kube-api-access-gr472\" (UniqueName: \"kubernetes.io/projected/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-kube-api-access-gr472\") pod \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\" (UID: \"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3\") "
Oct 30 22:15:27.248874 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.248846    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955a3c91-6234-4799-8656-0c2596f708de-plugins-config-volume" pod "955a3c91-6234-4799-8656-0c2596f708de" (UID: "955a3c91-6234-4799-8656-0c2596f708de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:27.248966 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.248914    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-plugins-config-volume" pod "44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" (UID: "44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:27.250674 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.250654    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-kube-api-access-gr472" pod "44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" (UID: "44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3"). InnerVolumeSpecName "kube-api-access-gr472". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:27.250738 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.250667    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955a3c91-6234-4799-8656-0c2596f708de-kube-api-access-khg4t" pod "955a3c91-6234-4799-8656-0c2596f708de" (UID: "955a3c91-6234-4799-8656-0c2596f708de"). InnerVolumeSpecName "kube-api-access-khg4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:27.350172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.350143    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955a3c91-6234-4799-8656-0c2596f708de-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:27.350172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.350167    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khg4t\" (UniqueName: \"kubernetes.io/projected/955a3c91-6234-4799-8656-0c2596f708de-kube-api-access-khg4t\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:27.350172 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.350179    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:27.350420 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.350188    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr472\" (UniqueName: \"kubernetes.io/projected/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3-kube-api-access-gr472\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:15:27.737996 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.737964    5817 generic.go:356] "Generic (PLEG): container finished" podID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" containerID="4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3" exitCode=0
Oct 30 22:15:27.738153 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.738011    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"
Oct 30 22:15:27.738153 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.738046    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" event={"ID":"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3","Type":"ContainerDied","Data":"4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3"}
Oct 30 22:15:27.738153 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.738098    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8" event={"ID":"44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3","Type":"ContainerDied","Data":"04ee3088705e3c5e9815b6121215c464e484d8077156fc33db3bab14dcbb754f"}
Oct 30 22:15:27.738153 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.738121    5817 scope.go:122] "RemoveContainer" containerID="4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3"
Oct 30 22:15:27.739329 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.739307    5817 generic.go:356] "Generic (PLEG): container finished" podID="955a3c91-6234-4799-8656-0c2596f708de" containerID="ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba" exitCode=0
Oct 30 22:15:27.739469 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.739370    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" event={"ID":"955a3c91-6234-4799-8656-0c2596f708de","Type":"ContainerDied","Data":"ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba"}
Oct 30 22:15:27.739469 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.739376    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"
Oct 30 22:15:27.739469 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.739400    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf" event={"ID":"955a3c91-6234-4799-8656-0c2596f708de","Type":"ContainerDied","Data":"ef12a5a41a6e1d014981fd77f7bd416e3c4bb3d49264c54ecb7c530c7ca37faf"}
Oct 30 22:15:27.747246 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.747142    5817 scope.go:122] "RemoveContainer" containerID="4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3"
Oct 30 22:15:27.747525 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:27.747482    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3\": container with ID starting with 4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3 not found: ID does not exist" containerID="4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3"
Oct 30 22:15:27.747525 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.747517    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3"} err="failed to get container status \"4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3\": rpc error: code = NotFound desc = could not find container \"4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3\": container with ID starting with 4ac7bc0a160a27b78c769707b1910b9c3a3eb78340811bcd69a63b9075ebf4b3 not found: ID does not exist"
Oct 30 22:15:27.747719 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.747535    5817 scope.go:122] "RemoveContainer" containerID="ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba"
Oct 30 22:15:27.755597 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.755577    5817 scope.go:122] "RemoveContainer" containerID="ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba"
Oct 30 22:15:27.755857 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:27.755840    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba\": container with ID starting with ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba not found: ID does not exist" containerID="ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba"
Oct 30 22:15:27.755897 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.755865    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba"} err="failed to get container status \"ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba\": rpc error: code = NotFound desc = could not find container \"ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba\": container with ID starting with ab0e8ddd97cd27a1fb0efcb7bb1533d99d698efcb885a58e9989cb1d954b7bba not found: ID does not exist"
Oct 30 22:15:27.906632 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.906603    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"]
Oct 30 22:15:27.953742 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:27.953718    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-788e42-68c655wwf8"]
Oct 30 22:15:28.017152 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:28.017127    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"]
Oct 30 22:15:28.021044 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:28.021021    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-788e42-c768cb948fvhpf"]
Oct 30 22:15:28.842737 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:28.842705    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" path="/var/lib/kubelet/pods/44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3/volumes"
Oct 30 22:15:28.843144 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:28.843128    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="955a3c91-6234-4799-8656-0c2596f708de" path="/var/lib/kubelet/pods/955a3c91-6234-4799-8656-0c2596f708de/volumes"
Oct 30 22:15:32.357370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:32.354399    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-jsclw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:15:32.357370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:15:32.354487    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-jsclw" podUID="7d34de55-a99d-4f02-b45e-915395b7d0b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:15:46.336640 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:15:46.336604    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:16:16.337921 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:16:16.337835    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:16:46.337702 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:16:46.337659    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:17:16.351290 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:17:16.351246    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:17:46.338121 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:17:46.338080    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:18:02.570295 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570262    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"]
Oct 30 22:18:02.570864 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570841    5817 cpu_manager.go:414] "containerMap: removing container" podUID="955a3c91-6234-4799-8656-0c2596f708de" containerName="payload-processing"
Oct 30 22:18:02.570864 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570858    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="955a3c91-6234-4799-8656-0c2596f708de" containerName="payload-processing"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570880    5817 cpu_manager.go:414] "containerMap: removing container" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570886    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570895    5817 cpu_manager.go:414] "containerMap: removing container" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570900    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570907    5817 cpu_manager.go:414] "containerMap: removing container" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570911    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570928    5817 cpu_manager.go:414] "containerMap: removing container" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570932    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.570944 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570946    5817 cpu_manager.go:414] "containerMap: removing container" podUID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" containerName="payload-pre-processing"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570951    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" containerName="payload-pre-processing"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570958    5817 cpu_manager.go:414] "containerMap: removing container" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.570962    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.571025    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.571033    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc698b74-0a5d-4894-ad5a-4d2be1816a91" containerName="istio-proxy"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.571044    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.571051    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.571060    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="44f31ff5-dddc-4f1c-b1d3-c57bcd8a07c3" containerName="payload-pre-processing"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.571066    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb4099a6-8809-4b25-a60e-f851c6f2bff1" containerName="cleanup"
Oct 30 22:18:02.571216 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.571073    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="955a3c91-6234-4799-8656-0c2596f708de" containerName="payload-processing"
Oct 30 22:18:02.574072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.574053    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.624898 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.624863    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-3ec071-openshift-default-dockercfg-2zww8\""
Oct 30 22:18:02.635121 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635084    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-data\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635291 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635192    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-istio-token\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635291 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635265    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcr9x\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-kube-api-access-vcr9x\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635446 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635318    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-credential-socket\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635446 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635370    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-envoy\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635446 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635429    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc937cd-a879-4223-a613-b258074f8776-istio-podinfo\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635647 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635472    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-socket\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635647 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635549    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-certs\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.635647 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.635585    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc937cd-a879-4223-a613-b258074f8776-istiod-ca-cert\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.673599 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.673564    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"]
Oct 30 22:18:02.737042 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737001    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcr9x\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-kube-api-access-vcr9x\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737308 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737051    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-credential-socket\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737308 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737081    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-envoy\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737308 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737139    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc937cd-a879-4223-a613-b258074f8776-istio-podinfo\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737308 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737172    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-socket\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737308 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737233    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-certs\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737308 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737265    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc937cd-a879-4223-a613-b258074f8776-istiod-ca-cert\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737740 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737305    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-data\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.737740 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.737407    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-istio-token\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.738264 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.738239    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-credential-socket\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.738543 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.738304    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-data\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.738543 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.738489    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-socket\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.738788 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.738759    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-certs\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.739413 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.739386    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc937cd-a879-4223-a613-b258074f8776-istiod-ca-cert\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.740796 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.740772    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc937cd-a879-4223-a613-b258074f8776-istio-podinfo\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.742708 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.742680    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-envoy\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.796072 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.796034    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-istio-token\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.796307 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.796285    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcr9x\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-kube-api-access-vcr9x\") pod \"e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") " pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:02.887098 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:02.887009    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:03.316183 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.316158    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"]
Oct 30 22:18:03.319008 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:18:03.318980    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc937cd_a879_4223_a613_b258074f8776.slice/crio-12e786d64590c8e58f81243bc66475b563807a84a8f52ca326038021737a23fb WatchSource:0}: Error finding container 12e786d64590c8e58f81243bc66475b563807a84a8f52ca326038021737a23fb: Status 404 returned error can't find the container with id 12e786d64590c8e58f81243bc66475b563807a84a8f52ca326038021737a23fb
Oct 30 22:18:03.321327 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.321291    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:18:03.321418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.321381    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:18:03.321418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.321412    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:18:03.727325 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.727287    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" event={"ID":"6bc937cd-a879-4223-a613-b258074f8776","Type":"ContainerStarted","Data":"929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3"}
Oct 30 22:18:03.727735 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.727366    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" event={"ID":"6bc937cd-a879-4223-a613-b258074f8776","Type":"ContainerStarted","Data":"12e786d64590c8e58f81243bc66475b563807a84a8f52ca326038021737a23fb"}
Oct 30 22:18:03.843466 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.843407    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podStartSLOduration=1.843392008 podStartE2EDuration="1.843392008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:03.842543099 +0000 UTC m=+3819.130265648" watchObservedRunningTime="2026-10-30 22:18:03.843392008 +0000 UTC m=+3819.131114553"
Oct 30 22:18:03.887795 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:03.887769    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:04.892932 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:04.892897    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:05.739814 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:05.739776    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:05.745451 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:05.745420    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:10.722921 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.722883    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"]
Oct 30 22:18:10.726864 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.726842    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:10.780401 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.780371    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-3ec071\""
Oct 30 22:18:10.820089 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.819939    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l29k\" (UniqueName: \"kubernetes.io/projected/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-kube-api-access-4l29k\") pod \"payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:10.820769 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.820490    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:10.828635 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.828609    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-3d4429b6-dockercfg-64qwc\""
Oct 30 22:18:10.831226 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.831195    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"]
Oct 30 22:18:10.877050 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.877011    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"]
Oct 30 22:18:10.881015 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.880990    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:10.921821 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.921785    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:10.921972 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.921954    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l29k\" (UniqueName: \"kubernetes.io/projected/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-kube-api-access-4l29k\") pod \"payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:10.922515 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.922497    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:10.927969 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.927943    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"]
Oct 30 22:18:10.987252 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:10.987178    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l29k\" (UniqueName: \"kubernetes.io/projected/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-kube-api-access-4l29k\") pod \"payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:11.023222 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.023183    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqh8q\" (UniqueName: \"kubernetes.io/projected/75235f3f-b272-4f95-8c2d-c15205ef7578-kube-api-access-qqh8q\") pod \"payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.023424 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.023299    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75235f3f-b272-4f95-8c2d-c15205ef7578-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.040148 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.040113    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:11.123972 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.123936    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqh8q\" (UniqueName: \"kubernetes.io/projected/75235f3f-b272-4f95-8c2d-c15205ef7578-kube-api-access-qqh8q\") pod \"payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.124132 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.124022    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75235f3f-b272-4f95-8c2d-c15205ef7578-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.124748 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.124728    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75235f3f-b272-4f95-8c2d-c15205ef7578-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.182315 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.182286    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqh8q\" (UniqueName: \"kubernetes.io/projected/75235f3f-b272-4f95-8c2d-c15205ef7578-kube-api-access-qqh8q\") pod \"payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.194292 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.194268    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.415829 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.415803    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"]
Oct 30 22:18:11.417891 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:18:11.417834    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8a9a1e_7d56_400f_ba6a_a5bd0349957f.slice/crio-eb41281276b4de66b1b5c19d6958faf2fe2cf2cfdf14bbbc242c3647e4536358 WatchSource:0}: Error finding container eb41281276b4de66b1b5c19d6958faf2fe2cf2cfdf14bbbc242c3647e4536358: Status 404 returned error can't find the container with id eb41281276b4de66b1b5c19d6958faf2fe2cf2cfdf14bbbc242c3647e4536358
Oct 30 22:18:11.567128 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.567099    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"]
Oct 30 22:18:11.568807 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:18:11.568780    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75235f3f_b272_4f95_8c2d_c15205ef7578.slice/crio-1497d4c84b39df3f7fe013f3a0057aa6d92a8f2463a65140b96dba76465978f5 WatchSource:0}: Error finding container 1497d4c84b39df3f7fe013f3a0057aa6d92a8f2463a65140b96dba76465978f5: Status 404 returned error can't find the container with id 1497d4c84b39df3f7fe013f3a0057aa6d92a8f2463a65140b96dba76465978f5
Oct 30 22:18:11.767530 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.767497    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" event={"ID":"75235f3f-b272-4f95-8c2d-c15205ef7578","Type":"ContainerStarted","Data":"11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f"}
Oct 30 22:18:11.768057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.767549    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" event={"ID":"75235f3f-b272-4f95-8c2d-c15205ef7578","Type":"ContainerStarted","Data":"1497d4c84b39df3f7fe013f3a0057aa6d92a8f2463a65140b96dba76465978f5"}
Oct 30 22:18:11.768057 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.767594    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:11.769187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.769162    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" event={"ID":"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f","Type":"ContainerStarted","Data":"4b07a91637bc4dc3b482d32e31c28dea41f5364646457d55d07f914beec21e02"}
Oct 30 22:18:11.769313 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.769209    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" event={"ID":"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f","Type":"ContainerStarted","Data":"eb41281276b4de66b1b5c19d6958faf2fe2cf2cfdf14bbbc242c3647e4536358"}
Oct 30 22:18:11.769313 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.769254    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:11.878807 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:11.878691    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" podStartSLOduration=1.878676526 podStartE2EDuration="1.878676526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:11.878256444 +0000 UTC m=+3827.165978990" watchObservedRunningTime="2026-10-30 22:18:11.878676526 +0000 UTC m=+3827.166399074"
Oct 30 22:18:16.337200 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:18:16.337163    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:18:22.777598 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:22.777567    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:18:22.778094 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:22.777636    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:18:22.910469 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:22.904593    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" podStartSLOduration=12.904573943 podStartE2EDuration="12.904573943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:11.994284052 +0000 UTC m=+3827.282006597" watchObservedRunningTime="2026-10-30 22:18:22.904573943 +0000 UTC m=+3838.192296486"
Oct 30 22:18:31.879404 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:31.879341    5817 generic.go:356] "Generic (PLEG): container finished" podID="6bc937cd-a879-4223-a613-b258074f8776" containerID="929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3" exitCode=0
Oct 30 22:18:31.879879 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:31.879416    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" event={"ID":"6bc937cd-a879-4223-a613-b258074f8776","Type":"ContainerDied","Data":"929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3"}
Oct 30 22:18:31.879879 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:31.879799    5817 scope.go:122] "RemoveContainer" containerID="929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3"
Oct 30 22:18:31.881075 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:31.881046    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:18:31.881142 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:31.881116    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:18:31.881179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:31.881150    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:18:32.886736 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:32.886695    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" event={"ID":"6bc937cd-a879-4223-a613-b258074f8776","Type":"ContainerStarted","Data":"6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e"}
Oct 30 22:18:32.887201 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:32.887177    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:32.898532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:32.897452    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:18:32.898532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:32.897527    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 22:18:33.887558 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:33.887520    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:18:33.887976 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:33.887580    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 22:18:34.887947 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:34.887913    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:18:34.888331 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:34.887972    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 22:18:35.739947 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:35.739908    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:35.887629 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:35.887596    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:18:35.887802 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:35.887646    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 22:18:36.887734 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:36.887704    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:18:36.888116 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:36.887753    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 22:18:37.887456 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:37.887409    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 22:18:37.887634 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:37.887478    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 22:18:38.891435 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:38.891399    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:38.892544 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:18:38.892520    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:18:46.337252 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:18:46.337218    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:19:16.337527 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:19:16.337445    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:19:38.989032 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:38.988996    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"]
Oct 30 22:19:39.149025 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:39.148988    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"]
Oct 30 22:19:43.990000 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:43.989958    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" podUID="75235f3f-b272-4f95-8c2d-c15205ef7578" containerName="payload-processing" containerID="cri-o://11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f" gracePeriod=25
Oct 30 22:19:44.149435 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:44.149395    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" podUID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" containerName="payload-pre-processing" containerID="cri-o://4b07a91637bc4dc3b482d32e31c28dea41f5364646457d55d07f914beec21e02" gracePeriod=25
Oct 30 22:19:46.336915 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:19:46.336868    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:19:52.776076 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:52.776031    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" podUID="75235f3f-b272-4f95-8c2d-c15205ef7578" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:19:52.776076 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:52.776063    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" podUID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:19:53.486075 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:53.486032    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"]
Oct 30 22:19:53.486305 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:53.486254    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" containerID="cri-o://6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e" gracePeriod=30
Oct 30 22:19:53.894495 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:53.894396    5817 patch_prober.go:29] interesting pod/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:19:53.894495 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:53.894467    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:19:58.753558 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.753501    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:19:58.825240 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825211    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6bc937cd-a879-4223-a613-b258074f8776-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc937cd-a879-4223-a613-b258074f8776-istio-podinfo\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825389 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825257    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6bc937cd-a879-4223-a613-b258074f8776-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc937cd-a879-4223-a613-b258074f8776-istiod-ca-cert\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825389 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825300    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-envoy\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825389 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825324    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-socket\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825389 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825381    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-data\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825582 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825421    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-kube-api-access-vcr9x\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-kube-api-access-vcr9x\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825636 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825622    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-credential-socket\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825693    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-certs\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.825936 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825755    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-socket" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:58.826088 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825922    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-istio-token\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-istio-token\") pod \"6bc937cd-a879-4223-a613-b258074f8776\" (UID: \"6bc937cd-a879-4223-a613-b258074f8776\") "
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825837    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-data" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825968    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-certs" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825960    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bc937cd-a879-4223-a613-b258074f8776-istiod-ca-cert" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.825939    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-credential-socket" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.826596    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.826621    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.826644    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc937cd-a879-4223-a613-b258074f8776-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.826668    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.826706 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.826683    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.827559 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.827533    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bc937cd-a879-4223-a613-b258074f8776-istio-podinfo" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:19:58.827658 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.827640    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-envoy" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:58.827915 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.827891    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-kube-api-access-vcr9x" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "kube-api-access-vcr9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:58.828165 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.828147    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-istio-token" pod "6bc937cd-a879-4223-a613-b258074f8776" (UID: "6bc937cd-a879-4223-a613-b258074f8776"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:58.928014 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.927979    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.928014 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.928013    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc937cd-a879-4223-a613-b258074f8776-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.928206 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.928027    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc937cd-a879-4223-a613-b258074f8776-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:58.928206 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:58.928039    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcr9x\" (UniqueName: \"kubernetes.io/projected/6bc937cd-a879-4223-a613-b258074f8776-kube-api-access-vcr9x\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:59.358175 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.358136    5817 generic.go:356] "Generic (PLEG): container finished" podID="6bc937cd-a879-4223-a613-b258074f8776" containerID="6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e" exitCode=0
Oct 30 22:19:59.358403 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.358197    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"
Oct 30 22:19:59.358403 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.358216    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" event={"ID":"6bc937cd-a879-4223-a613-b258074f8776","Type":"ContainerDied","Data":"6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e"}
Oct 30 22:19:59.358403 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.358302    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85" event={"ID":"6bc937cd-a879-4223-a613-b258074f8776","Type":"ContainerDied","Data":"12e786d64590c8e58f81243bc66475b563807a84a8f52ca326038021737a23fb"}
Oct 30 22:19:59.358403 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.358325    5817 scope.go:122] "RemoveContainer" containerID="6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e"
Oct 30 22:19:59.367432 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.367408    5817 scope.go:122] "RemoveContainer" containerID="929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3"
Oct 30 22:19:59.378240 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.378219    5817 scope.go:122] "RemoveContainer" containerID="6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e"
Oct 30 22:19:59.378527 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:19:59.378508    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e\": container with ID starting with 6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e not found: ID does not exist" containerID="6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e"
Oct 30 22:19:59.378590 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.378536    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e"} err="failed to get container status \"6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e\": rpc error: code = NotFound desc = could not find container \"6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e\": container with ID starting with 6f8ee778fb2e03656092c59a3fac8c5f855bc340b37682d1c5fb512f85161e4e not found: ID does not exist"
Oct 30 22:19:59.378590 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.378554    5817 scope.go:122] "RemoveContainer" containerID="929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3"
Oct 30 22:19:59.378792 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:19:59.378774    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3\": container with ID starting with 929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3 not found: ID does not exist" containerID="929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3"
Oct 30 22:19:59.378835 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.378797    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3"} err="failed to get container status \"929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3\": rpc error: code = NotFound desc = could not find container \"929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3\": container with ID starting with 929304822b2e75174dff1393eb1c0fcce47071340e163455f43e2fa97b69f1e3 not found: ID does not exist"
Oct 30 22:19:59.514391 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.514354    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"]
Oct 30 22:19:59.561418 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:19:59.561386    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3ec071-openshift-default-6c569c8f57n7p85"]
Oct 30 22:20:00.842442 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:00.842413    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6bc937cd-a879-4223-a613-b258074f8776" path="/var/lib/kubelet/pods/6bc937cd-a879-4223-a613-b258074f8776/volumes"
Oct 30 22:20:02.776238 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:02.776204    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" podUID="75235f3f-b272-4f95-8c2d-c15205ef7578" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:20:02.776636 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:02.776210    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" podUID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:20:04.281458 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.281434    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:20:04.380025 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.379992    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75235f3f-b272-4f95-8c2d-c15205ef7578-kube-api-access-qqh8q\" (UniqueName: \"kubernetes.io/projected/75235f3f-b272-4f95-8c2d-c15205ef7578-kube-api-access-qqh8q\") pod \"75235f3f-b272-4f95-8c2d-c15205ef7578\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") "
Oct 30 22:20:04.380202 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.380081    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75235f3f-b272-4f95-8c2d-c15205ef7578-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75235f3f-b272-4f95-8c2d-c15205ef7578-plugins-config-volume\") pod \"75235f3f-b272-4f95-8c2d-c15205ef7578\" (UID: \"75235f3f-b272-4f95-8c2d-c15205ef7578\") "
Oct 30 22:20:04.380528 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.380502    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75235f3f-b272-4f95-8c2d-c15205ef7578-plugins-config-volume" pod "75235f3f-b272-4f95-8c2d-c15205ef7578" (UID: "75235f3f-b272-4f95-8c2d-c15205ef7578"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:04.382324 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.382298    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75235f3f-b272-4f95-8c2d-c15205ef7578-kube-api-access-qqh8q" pod "75235f3f-b272-4f95-8c2d-c15205ef7578" (UID: "75235f3f-b272-4f95-8c2d-c15205ef7578"). InnerVolumeSpecName "kube-api-access-qqh8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:04.386689 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.386657    5817 generic.go:356] "Generic (PLEG): container finished" podID="75235f3f-b272-4f95-8c2d-c15205ef7578" containerID="11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f" exitCode=0
Oct 30 22:20:04.386804 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.386716    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"
Oct 30 22:20:04.386804 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.386715    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" event={"ID":"75235f3f-b272-4f95-8c2d-c15205ef7578","Type":"ContainerDied","Data":"11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f"}
Oct 30 22:20:04.386931 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.386813    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv" event={"ID":"75235f3f-b272-4f95-8c2d-c15205ef7578","Type":"ContainerDied","Data":"1497d4c84b39df3f7fe013f3a0057aa6d92a8f2463a65140b96dba76465978f5"}
Oct 30 22:20:04.386931 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.386866    5817 scope.go:122] "RemoveContainer" containerID="11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f"
Oct 30 22:20:04.391309 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.391213    5817 generic.go:356] "Generic (PLEG): container finished" podID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" containerID="4b07a91637bc4dc3b482d32e31c28dea41f5364646457d55d07f914beec21e02" exitCode=0
Oct 30 22:20:04.391309 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.391258    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" event={"ID":"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f","Type":"ContainerDied","Data":"4b07a91637bc4dc3b482d32e31c28dea41f5364646457d55d07f914beec21e02"}
Oct 30 22:20:04.399581 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.399560    5817 scope.go:122] "RemoveContainer" containerID="11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f"
Oct 30 22:20:04.400034 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:20:04.400011    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f\": container with ID starting with 11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f not found: ID does not exist" containerID="11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f"
Oct 30 22:20:04.400137 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.400040    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f"} err="failed to get container status \"11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f\": rpc error: code = NotFound desc = could not find container \"11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f\": container with ID starting with 11fe1a42db0007d5367a28e03eee89d66b14bb5fdf6f578c8585b3252a06666f not found: ID does not exist"
Oct 30 22:20:04.412437 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.412415    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:20:04.481291 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.481256    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-kube-api-access-4l29k\" (UniqueName: \"kubernetes.io/projected/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-kube-api-access-4l29k\") pod \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") "
Oct 30 22:20:04.481467 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.481315    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-plugins-config-volume\") pod \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\" (UID: \"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f\") "
Oct 30 22:20:04.481634 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.481620    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqh8q\" (UniqueName: \"kubernetes.io/projected/75235f3f-b272-4f95-8c2d-c15205ef7578-kube-api-access-qqh8q\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:04.481682 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.481637    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75235f3f-b272-4f95-8c2d-c15205ef7578-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:04.481717 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.481691    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-plugins-config-volume" pod "1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" (UID: "1e8a9a1e-7d56-400f-ba6a-a5bd0349957f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:04.483433 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.483414    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-kube-api-access-4l29k" pod "1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" (UID: "1e8a9a1e-7d56-400f-ba6a-a5bd0349957f"). InnerVolumeSpecName "kube-api-access-4l29k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:04.582976 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.582900    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l29k\" (UniqueName: \"kubernetes.io/projected/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-kube-api-access-4l29k\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:04.582976 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.582927    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:04.663952 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.663916    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"]
Oct 30 22:20:04.668359 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.668319    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3ec071-6749b77d4qwnkv"]
Oct 30 22:20:04.842359 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:04.842268    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75235f3f-b272-4f95-8c2d-c15205ef7578" path="/var/lib/kubelet/pods/75235f3f-b272-4f95-8c2d-c15205ef7578/volumes"
Oct 30 22:20:05.403371 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:05.403056    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp" event={"ID":"1e8a9a1e-7d56-400f-ba6a-a5bd0349957f","Type":"ContainerDied","Data":"eb41281276b4de66b1b5c19d6958faf2fe2cf2cfdf14bbbc242c3647e4536358"}
Oct 30 22:20:05.403371 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:05.403094    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"
Oct 30 22:20:05.403371 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:05.403143    5817 scope.go:122] "RemoveContainer" containerID="4b07a91637bc4dc3b482d32e31c28dea41f5364646457d55d07f914beec21e02"
Oct 30 22:20:05.562990 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:05.562954    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"]
Oct 30 22:20:05.610801 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:05.609868    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ec071-55dc8vknjp"]
Oct 30 22:20:06.877390 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:06.876816    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" path="/var/lib/kubelet/pods/1e8a9a1e-7d56-400f-ba6a-a5bd0349957f/volumes"
Oct 30 22:20:16.337228 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:20:16.337189    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:20:42.923547 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.923512    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"]
Oct 30 22:20:42.923973 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.923955    5817 cpu_manager.go:414] "containerMap: removing container" podUID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" containerName="payload-pre-processing"
Oct 30 22:20:42.924029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.923975    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" containerName="payload-pre-processing"
Oct 30 22:20:42.924029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924000    5817 cpu_manager.go:414] "containerMap: removing container" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy"
Oct 30 22:20:42.924029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924006    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy"
Oct 30 22:20:42.924029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924020    5817 cpu_manager.go:414] "containerMap: removing container" podUID="75235f3f-b272-4f95-8c2d-c15205ef7578" containerName="payload-processing"
Oct 30 22:20:42.924029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924025    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75235f3f-b272-4f95-8c2d-c15205ef7578" containerName="payload-processing"
Oct 30 22:20:42.924187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924039    5817 cpu_manager.go:414] "containerMap: removing container" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy"
Oct 30 22:20:42.924187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924047    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy"
Oct 30 22:20:42.924187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924124    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy"
Oct 30 22:20:42.924187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924135    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="75235f3f-b272-4f95-8c2d-c15205ef7578" containerName="payload-processing"
Oct 30 22:20:42.924187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.924147    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e8a9a1e-7d56-400f-ba6a-a5bd0349957f" containerName="payload-pre-processing"
Oct 30 22:20:42.928580 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:42.928560    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.022911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.022876    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-tmp-dir\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.023088 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.022925    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz2wh\" (UniqueName: \"kubernetes.io/projected/e3439250-7014-48c7-acc6-99fba203bae2-kube-api-access-pz2wh\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.023088 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.023016    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-dshm\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.023218 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.023142    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3439250-7014-48c7-acc6-99fba203bae2-tls-certs\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.023218 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.023185    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-model-cache\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.023326 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.023232    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-home\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.024879 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.024858    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-38871958-kserve-self-signed-certs\""
Oct 30 22:20:43.026681 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.026662    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"]
Oct 30 22:20:43.124781 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.124750    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-dshm\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.124961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.124792    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3439250-7014-48c7-acc6-99fba203bae2-tls-certs\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.124961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.124811    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-model-cache\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.124961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.124846    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-home\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.124961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.124943    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-tmp-dir\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.125144 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.124992    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz2wh\" (UniqueName: \"kubernetes.io/projected/e3439250-7014-48c7-acc6-99fba203bae2-kube-api-access-pz2wh\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.125338 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.125298    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-model-cache\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.125464 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.125341    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-tmp-dir\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.125464 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.125389    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-home\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.127141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.127119    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-dshm\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.127304 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.127285    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3439250-7014-48c7-acc6-99fba203bae2-tls-certs\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.183767 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.183709    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz2wh\" (UniqueName: \"kubernetes.io/projected/e3439250-7014-48c7-acc6-99fba203bae2-kube-api-access-pz2wh\") pod \"e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") " pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.239770 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.239740    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:43.605855 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.605831    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"]
Oct 30 22:20:43.608255 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:20:43.608223    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3439250_7014_48c7_acc6_99fba203bae2.slice/crio-5a32377fc6b444f8c845a11041f816b6f4426835459770ff02caf3c5596cd042 WatchSource:0}: Error finding container 5a32377fc6b444f8c845a11041f816b6f4426835459770ff02caf3c5596cd042: Status 404 returned error can't find the container with id 5a32377fc6b444f8c845a11041f816b6f4426835459770ff02caf3c5596cd042
Oct 30 22:20:43.628608 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:43.628582    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56" event={"ID":"e3439250-7014-48c7-acc6-99fba203bae2","Type":"ContainerStarted","Data":"5a32377fc6b444f8c845a11041f816b6f4426835459770ff02caf3c5596cd042"}
Oct 30 22:20:44.634408 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:44.634368    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56" event={"ID":"e3439250-7014-48c7-acc6-99fba203bae2","Type":"ContainerStarted","Data":"ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6"}
Oct 30 22:20:44.747007 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:44.746961    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56" podStartSLOduration=2.746948367 podStartE2EDuration="2.746948367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:44.746072758 +0000 UTC m=+3980.033795309" watchObservedRunningTime="2026-10-30 22:20:44.746948367 +0000 UTC m=+3980.034670941"
Oct 30 22:20:45.240376 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:45.240325    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:46.337340 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:20:46.337307    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:20:53.240844 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:53.240810    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:53.253077 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:53.253050    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:20:53.752363 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:20:53.752320    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:21:16.337051 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:21:16.337008    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:21:46.336811 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:21:46.336771    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:21:48.919372 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:21:48.915450    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"]
Oct 30 22:22:03.939504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:03.939414    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56" podUID="e3439250-7014-48c7-acc6-99fba203bae2" containerName="main" containerID="cri-o://ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6" gracePeriod=45
Oct 30 22:22:04.208129 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.208106    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:22:04.224316 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.224284    5817 generic.go:356] "Generic (PLEG): container finished" podID="e3439250-7014-48c7-acc6-99fba203bae2" containerID="ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6" exitCode=0
Oct 30 22:22:04.224445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.224338    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"
Oct 30 22:22:04.224445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.224371    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56" event={"ID":"e3439250-7014-48c7-acc6-99fba203bae2","Type":"ContainerDied","Data":"ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6"}
Oct 30 22:22:04.224445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.224410    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56" event={"ID":"e3439250-7014-48c7-acc6-99fba203bae2","Type":"ContainerDied","Data":"5a32377fc6b444f8c845a11041f816b6f4426835459770ff02caf3c5596cd042"}
Oct 30 22:22:04.224445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.224428    5817 scope.go:122] "RemoveContainer" containerID="ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6"
Oct 30 22:22:04.232821 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.232790    5817 scope.go:122] "RemoveContainer" containerID="ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6"
Oct 30 22:22:04.233077 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:22:04.233058    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6\": container with ID starting with ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6 not found: ID does not exist" containerID="ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6"
Oct 30 22:22:04.233146 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.233085    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6"} err="failed to get container status \"ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6\": rpc error: code = NotFound desc = could not find container \"ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6\": container with ID starting with ad08cc893eec24c64c3a813686ec49851e172e6e5359691674f098fed182b8b6 not found: ID does not exist"
Oct 30 22:22:04.332679 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.332651    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-home\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-home\") pod \"e3439250-7014-48c7-acc6-99fba203bae2\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") "
Oct 30 22:22:04.332833 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.332730    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-model-cache\") pod \"e3439250-7014-48c7-acc6-99fba203bae2\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") "
Oct 30 22:22:04.332833 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.332785    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-tmp-dir\") pod \"e3439250-7014-48c7-acc6-99fba203bae2\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") "
Oct 30 22:22:04.332833 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.332810    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3439250-7014-48c7-acc6-99fba203bae2-tls-certs\" (UniqueName: \"kubernetes.io/secret/e3439250-7014-48c7-acc6-99fba203bae2-tls-certs\") pod \"e3439250-7014-48c7-acc6-99fba203bae2\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") "
Oct 30 22:22:04.333018 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.332853    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3439250-7014-48c7-acc6-99fba203bae2-kube-api-access-pz2wh\" (UniqueName: \"kubernetes.io/projected/e3439250-7014-48c7-acc6-99fba203bae2-kube-api-access-pz2wh\") pod \"e3439250-7014-48c7-acc6-99fba203bae2\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") "
Oct 30 22:22:04.333018 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.332879    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-dshm\") pod \"e3439250-7014-48c7-acc6-99fba203bae2\" (UID: \"e3439250-7014-48c7-acc6-99fba203bae2\") "
Oct 30 22:22:04.333018 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.332940    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-home" pod "e3439250-7014-48c7-acc6-99fba203bae2" (UID: "e3439250-7014-48c7-acc6-99fba203bae2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:04.333166 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.333042    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-model-cache" pod "e3439250-7014-48c7-acc6-99fba203bae2" (UID: "e3439250-7014-48c7-acc6-99fba203bae2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:04.333166 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.333086    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-tmp-dir" pod "e3439250-7014-48c7-acc6-99fba203bae2" (UID: "e3439250-7014-48c7-acc6-99fba203bae2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:04.333265 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.333247    5817 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-model-cache\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:04.333320 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.333269    5817 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-tmp-dir\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:04.333320 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.333284    5817 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-home\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:04.335077 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.335056    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-dshm" pod "e3439250-7014-48c7-acc6-99fba203bae2" (UID: "e3439250-7014-48c7-acc6-99fba203bae2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:04.335506 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.335469    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3439250-7014-48c7-acc6-99fba203bae2-tls-certs" pod "e3439250-7014-48c7-acc6-99fba203bae2" (UID: "e3439250-7014-48c7-acc6-99fba203bae2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:04.335594 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.335543    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3439250-7014-48c7-acc6-99fba203bae2-kube-api-access-pz2wh" pod "e3439250-7014-48c7-acc6-99fba203bae2" (UID: "e3439250-7014-48c7-acc6-99fba203bae2"). InnerVolumeSpecName "kube-api-access-pz2wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:04.434302 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.434273    5817 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3439250-7014-48c7-acc6-99fba203bae2-tls-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:04.434302 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.434301    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz2wh\" (UniqueName: \"kubernetes.io/projected/e3439250-7014-48c7-acc6-99fba203bae2-kube-api-access-pz2wh\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:04.434479 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.434311    5817 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3439250-7014-48c7-acc6-99fba203bae2-dshm\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:04.639256 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.639223    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"]
Oct 30 22:22:04.643361 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.643321    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-38871958-kserve-577b8bcd49-x2m56"]
Oct 30 22:22:04.843222 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:04.843133    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3439250-7014-48c7-acc6-99fba203bae2" path="/var/lib/kubelet/pods/e3439250-7014-48c7-acc6-99fba203bae2/volumes"
Oct 30 22:22:16.337670 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:22:16.337626    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:22:46.339561 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:22:46.339519    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:22:47.646859 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.646824    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"]
Oct 30 22:22:47.647550 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.647527    5817 cpu_manager.go:414] "containerMap: removing container" podUID="e3439250-7014-48c7-acc6-99fba203bae2" containerName="main"
Oct 30 22:22:47.647672 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.647550    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3439250-7014-48c7-acc6-99fba203bae2" containerName="main"
Oct 30 22:22:47.647734 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.647723    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3439250-7014-48c7-acc6-99fba203bae2" containerName="main"
Oct 30 22:22:47.647788 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.647739    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="6bc937cd-a879-4223-a613-b258074f8776" containerName="istio-proxy"
Oct 30 22:22:47.650852 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.650833    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.705179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.705123    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8c501563-openshift-default-dockercfg-2jzk8\""
Oct 30 22:22:47.753616 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.753581    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"]
Oct 30 22:22:47.754039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754012    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05264fea-2fe9-4e27-9d1a-b667c43add57-istiod-ca-cert\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754149 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754129    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-podinfo\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754220 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754200    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-socket\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754272 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754249    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-credential-socket\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754319 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754273    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ww29\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-kube-api-access-5ww29\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754424 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754407    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-certs\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754495 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754451    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-envoy\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754546 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754527    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-token\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.754626 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.754611    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-data\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856192 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856161    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-certs\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856412 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856199    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-envoy\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856412 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856220    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-token\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856412 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856265    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-data\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856412 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856298    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05264fea-2fe9-4e27-9d1a-b667c43add57-istiod-ca-cert\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856412 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856341    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-podinfo\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856480    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-socket\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856519    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-credential-socket\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856552    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ww29\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-kube-api-access-5ww29\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856707 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856660    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-certs\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856751    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-data\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856840    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-socket\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.856938 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.856917    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-credential-socket\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.857110 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.857047    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05264fea-2fe9-4e27-9d1a-b667c43add57-istiod-ca-cert\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.858641 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.858616    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-envoy\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.858910 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.858893    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-podinfo\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.917312 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.917253    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ww29\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-kube-api-access-5ww29\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.917703 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.917685    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-token\") pod \"e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") " pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:47.964144 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:47.964109    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:48.399943 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:22:48.399906    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05264fea_2fe9_4e27_9d1a_b667c43add57.slice/crio-5880f7d5095e987aa1abec3a0967b2873cb57fd42c7d7f5beeb73b3c9a3bc420 WatchSource:0}: Error finding container 5880f7d5095e987aa1abec3a0967b2873cb57fd42c7d7f5beeb73b3c9a3bc420: Status 404 returned error can't find the container with id 5880f7d5095e987aa1abec3a0967b2873cb57fd42c7d7f5beeb73b3c9a3bc420
Oct 30 22:22:48.400093 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.399995    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"]
Oct 30 22:22:48.402554 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.402517    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:48.402949 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.402586    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:48.402949 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.402616    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:22:48.513290 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.513245    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" event={"ID":"05264fea-2fe9-4e27-9d1a-b667c43add57","Type":"ContainerStarted","Data":"12e8e8a9aa645e5f247ecccdaffebbf68483ec8371a0641804d89531b5cd0493"}
Oct 30 22:22:48.513502 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.513319    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" event={"ID":"05264fea-2fe9-4e27-9d1a-b667c43add57","Type":"ContainerStarted","Data":"5880f7d5095e987aa1abec3a0967b2873cb57fd42c7d7f5beeb73b3c9a3bc420"}
Oct 30 22:22:48.635202 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.635135    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podStartSLOduration=1.635117205 podStartE2EDuration="1.635117205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:48.634061701 +0000 UTC m=+4103.921784273" watchObservedRunningTime="2026-10-30 22:22:48.635117205 +0000 UTC m=+4103.922839752"
Oct 30 22:22:48.964437 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:48.964400    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:49.183031 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.182996    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"]
Oct 30 22:22:49.186797 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.186772    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.269595 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.269525    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b8b92c4-8372-4f36-951e-31db7f96cc47-maas-api-tls\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.269752 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.269604    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58wwd\" (UniqueName: \"kubernetes.io/projected/4b8b92c4-8372-4f36-951e-31db7f96cc47-kube-api-access-58wwd\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.269752 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.269688    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b8b92c4-8372-4f36-951e-31db7f96cc47-metrics-certs\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.269752 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.269716    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b92c4-8372-4f36-951e-31db7f96cc47-openshift-service-ca\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.284736 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.284684    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"]
Oct 30 22:22:49.370943 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.370910    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b8b92c4-8372-4f36-951e-31db7f96cc47-metrics-certs\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.371097 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.370953    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b92c4-8372-4f36-951e-31db7f96cc47-openshift-service-ca\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.371097 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.371029    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b8b92c4-8372-4f36-951e-31db7f96cc47-maas-api-tls\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.371097 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.371049    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58wwd\" (UniqueName: \"kubernetes.io/projected/4b8b92c4-8372-4f36-951e-31db7f96cc47-kube-api-access-58wwd\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.371907 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.371880    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b8b92c4-8372-4f36-951e-31db7f96cc47-openshift-service-ca\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.373527 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.373502    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b8b92c4-8372-4f36-951e-31db7f96cc47-metrics-certs\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.373615 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.373578    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b8b92c4-8372-4f36-951e-31db7f96cc47-maas-api-tls\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.428695 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.428663    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58wwd\" (UniqueName: \"kubernetes.io/projected/4b8b92c4-8372-4f36-951e-31db7f96cc47-kube-api-access-58wwd\") pod \"maas-api-6687d569c9-6vwg5\" (UID: \"4b8b92c4-8372-4f36-951e-31db7f96cc47\") " pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.500518 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.500483    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:49.882302 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.882173    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"]
Oct 30 22:22:49.885236 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:22:49.885206    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8b92c4_8372_4f36_951e_31db7f96cc47.slice/crio-1986f1a72195bad1e2e6eba1e202fe969fdd4a3b55a28559b058691d66fcc650 WatchSource:0}: Error finding container 1986f1a72195bad1e2e6eba1e202fe969fdd4a3b55a28559b058691d66fcc650: Status 404 returned error can't find the container with id 1986f1a72195bad1e2e6eba1e202fe969fdd4a3b55a28559b058691d66fcc650
Oct 30 22:22:49.886699 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.886680    5817 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:22:49.969187 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:49.969159    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:50.525563 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:50.525527    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5" event={"ID":"4b8b92c4-8372-4f36-951e-31db7f96cc47","Type":"ContainerStarted","Data":"1986f1a72195bad1e2e6eba1e202fe969fdd4a3b55a28559b058691d66fcc650"}
Oct 30 22:22:50.525852 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:50.525835    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:50.526687 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:50.526665    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:22:51.532320 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:51.532290    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5" event={"ID":"4b8b92c4-8372-4f36-951e-31db7f96cc47","Type":"ContainerStarted","Data":"15d2002d72ae8439463d7c0018ecbcdd7a34ed55ab721507cd838f801117ba11"}
Oct 30 22:22:51.532821 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:51.532522    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:51.654536 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:51.654485    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5" podStartSLOduration=1.735115049 podStartE2EDuration="2.654471051s" totalImagesPullingTime="919.356002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:51.652195364 +0000 UTC m=+4106.939917910" watchObservedRunningTime="2026-10-30 22:22:51.654471051 +0000 UTC m=+4106.942193637"
Oct 30 22:22:54.694196 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.694163    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"]
Oct 30 22:22:54.698334 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.698313    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:54.725439 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.725416    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnwvd\" (UniqueName: \"kubernetes.io/projected/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-kube-api-access-pnwvd\") pod \"payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:54.725603 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.725577    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:54.752798 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.752770    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"]
Oct 30 22:22:54.794484 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.794450    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8c501563\""
Oct 30 22:22:54.794732 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.794579    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8c501563-dockercfg-btg4t\""
Oct 30 22:22:54.826200 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.826164    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:54.826424 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.826251    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnwvd\" (UniqueName: \"kubernetes.io/projected/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-kube-api-access-pnwvd\") pod \"payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:54.826961 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.826932    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:54.843077 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.843053    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"]
Oct 30 22:22:54.847059 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.847035    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:54.891884 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.891852    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnwvd\" (UniqueName: \"kubernetes.io/projected/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-kube-api-access-pnwvd\") pod \"payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:54.892557 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.892538    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"]
Oct 30 22:22:54.927160 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.927125    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9863cecd-209e-420f-a311-028fc20d27b8-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c501563-7679965476-c46rf\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") " pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:54.927281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:54.927229    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxfwf\" (UniqueName: \"kubernetes.io/projected/9863cecd-209e-420f-a311-028fc20d27b8-kube-api-access-wxfwf\") pod \"payload-processing-e2e-ait-8c501563-7679965476-c46rf\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") " pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:55.011024 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.010992    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:55.028286 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.028258    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9863cecd-209e-420f-a311-028fc20d27b8-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c501563-7679965476-c46rf\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") " pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:55.028445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.028387    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxfwf\" (UniqueName: \"kubernetes.io/projected/9863cecd-209e-420f-a311-028fc20d27b8-kube-api-access-wxfwf\") pod \"payload-processing-e2e-ait-8c501563-7679965476-c46rf\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") " pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:55.029150 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.029112    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9863cecd-209e-420f-a311-028fc20d27b8-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c501563-7679965476-c46rf\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") " pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:55.097151 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.097114    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxfwf\" (UniqueName: \"kubernetes.io/projected/9863cecd-209e-420f-a311-028fc20d27b8-kube-api-access-wxfwf\") pod \"payload-processing-e2e-ait-8c501563-7679965476-c46rf\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") " pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:55.160437 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.160413    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:55.377336 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.377308    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"]
Oct 30 22:22:55.379614 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:22:55.379580    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7d0704_ac63_4391_b1cc_d6b0eb7b00b5.slice/crio-cff0a116c4dc80f51ca84f121bfff31488564647d2556541480450db4993531c WatchSource:0}: Error finding container cff0a116c4dc80f51ca84f121bfff31488564647d2556541480450db4993531c: Status 404 returned error can't find the container with id cff0a116c4dc80f51ca84f121bfff31488564647d2556541480450db4993531c
Oct 30 22:22:55.526639 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.526619    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"]
Oct 30 22:22:55.529003 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:22:55.528974    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9863cecd_209e_420f_a311_028fc20d27b8.slice/crio-fa1ad22f32b282a7f4eb9857826385ddefa15f5866032a03905309c106bea0bd WatchSource:0}: Error finding container fa1ad22f32b282a7f4eb9857826385ddefa15f5866032a03905309c106bea0bd: Status 404 returned error can't find the container with id fa1ad22f32b282a7f4eb9857826385ddefa15f5866032a03905309c106bea0bd
Oct 30 22:22:55.559856 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.559830    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9" event={"ID":"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5","Type":"ContainerStarted","Data":"f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441"}
Oct 30 22:22:55.559970 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.559885    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9" event={"ID":"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5","Type":"ContainerStarted","Data":"cff0a116c4dc80f51ca84f121bfff31488564647d2556541480450db4993531c"}
Oct 30 22:22:55.559970 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.559951    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:22:55.561250 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.561226    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf" event={"ID":"9863cecd-209e-420f-a311-028fc20d27b8","Type":"ContainerStarted","Data":"fa1ad22f32b282a7f4eb9857826385ddefa15f5866032a03905309c106bea0bd"}
Oct 30 22:22:55.670409 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:55.670278    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9" podStartSLOduration=1.670262269 podStartE2EDuration="1.670262269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:55.669476952 +0000 UTC m=+4110.957199498" watchObservedRunningTime="2026-10-30 22:22:55.670262269 +0000 UTC m=+4110.957984816"
Oct 30 22:22:56.568146 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:56.568107    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf" event={"ID":"9863cecd-209e-420f-a311-028fc20d27b8","Type":"ContainerStarted","Data":"63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2"}
Oct 30 22:22:56.683415 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:56.683339    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf" podStartSLOduration=2.683321965 podStartE2EDuration="2.683321965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:56.681771764 +0000 UTC m=+4111.969494311" watchObservedRunningTime="2026-10-30 22:22:56.683321965 +0000 UTC m=+4111.971044511"
Oct 30 22:22:57.543168 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:57.543139    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6687d569c9-6vwg5"
Oct 30 22:22:57.573231 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:57.573202    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:22:57.704304 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:57.704267    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"]
Oct 30 22:22:57.704602 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:57.704573    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2" podUID="72d7fcdd-2942-4884-ac28-12fd93421de7" containerName="maas-api" containerID="cri-o://32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23" gracePeriod=30
Oct 30 22:22:58.998435 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:58.998407    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 22:22:59.069431 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.069315    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72d7fcdd-2942-4884-ac28-12fd93421de7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d7fcdd-2942-4884-ac28-12fd93421de7-openshift-service-ca\") pod \"72d7fcdd-2942-4884-ac28-12fd93421de7\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") "
Oct 30 22:22:59.069431 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.069380    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-metrics-certs\") pod \"72d7fcdd-2942-4884-ac28-12fd93421de7\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") "
Oct 30 22:22:59.069654 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.069451    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-maas-api-tls\") pod \"72d7fcdd-2942-4884-ac28-12fd93421de7\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") "
Oct 30 22:22:59.069654 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.069489    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72d7fcdd-2942-4884-ac28-12fd93421de7-kube-api-access-6tkt7\" (UniqueName: \"kubernetes.io/projected/72d7fcdd-2942-4884-ac28-12fd93421de7-kube-api-access-6tkt7\") pod \"72d7fcdd-2942-4884-ac28-12fd93421de7\" (UID: \"72d7fcdd-2942-4884-ac28-12fd93421de7\") "
Oct 30 22:22:59.069776 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.069649    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72d7fcdd-2942-4884-ac28-12fd93421de7-openshift-service-ca" pod "72d7fcdd-2942-4884-ac28-12fd93421de7" (UID: "72d7fcdd-2942-4884-ac28-12fd93421de7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:22:59.069911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.069892    5817 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d7fcdd-2942-4884-ac28-12fd93421de7-openshift-service-ca\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:59.071514 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.071490    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-metrics-certs" pod "72d7fcdd-2942-4884-ac28-12fd93421de7" (UID: "72d7fcdd-2942-4884-ac28-12fd93421de7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:59.071603 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.071566    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72d7fcdd-2942-4884-ac28-12fd93421de7-kube-api-access-6tkt7" pod "72d7fcdd-2942-4884-ac28-12fd93421de7" (UID: "72d7fcdd-2942-4884-ac28-12fd93421de7"). InnerVolumeSpecName "kube-api-access-6tkt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:59.071658 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.071613    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-maas-api-tls" pod "72d7fcdd-2942-4884-ac28-12fd93421de7" (UID: "72d7fcdd-2942-4884-ac28-12fd93421de7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:59.171319 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.171288    5817 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-metrics-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:59.171319 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.171313    5817 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72d7fcdd-2942-4884-ac28-12fd93421de7-maas-api-tls\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:59.171319 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.171323    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tkt7\" (UniqueName: \"kubernetes.io/projected/72d7fcdd-2942-4884-ac28-12fd93421de7-kube-api-access-6tkt7\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:59.584820 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.584779    5817 generic.go:356] "Generic (PLEG): container finished" podID="72d7fcdd-2942-4884-ac28-12fd93421de7" containerID="32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23" exitCode=0
Oct 30 22:22:59.584994 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.584847    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"
Oct 30 22:22:59.584994 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.584865    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2" event={"ID":"72d7fcdd-2942-4884-ac28-12fd93421de7","Type":"ContainerDied","Data":"32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23"}
Oct 30 22:22:59.584994 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.584944    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2" event={"ID":"72d7fcdd-2942-4884-ac28-12fd93421de7","Type":"ContainerDied","Data":"ed12e0821b1ea7065a4df4b14b21938d9a8eb214ab4f4080e4d7f756f1858566"}
Oct 30 22:22:59.584994 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.584966    5817 scope.go:122] "RemoveContainer" containerID="32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23"
Oct 30 22:22:59.595318 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.595298    5817 scope.go:122] "RemoveContainer" containerID="32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23"
Oct 30 22:22:59.595616 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:22:59.595596    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23\": container with ID starting with 32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23 not found: ID does not exist" containerID="32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23"
Oct 30 22:22:59.595681 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.595624    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23"} err="failed to get container status \"32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23\": rpc error: code = NotFound desc = could not find container \"32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23\": container with ID starting with 32be97a39977d50387b519e91d9ce11d16fd860e4fd9336e799f1e840b470e23 not found: ID does not exist"
Oct 30 22:22:59.741271 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.741236    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"]
Oct 30 22:22:59.746779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:22:59.746751    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-86b5466f75-xbqs2"]
Oct 30 22:23:00.842251 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:00.842221    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72d7fcdd-2942-4884-ac28-12fd93421de7" path="/var/lib/kubelet/pods/72d7fcdd-2942-4884-ac28-12fd93421de7/volumes"
Oct 30 22:23:06.571275 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:06.571246    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:23:08.581425 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:08.581398    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:23:16.340461 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:23:16.340421    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:23:20.527059 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.527023    5817 patch_prober.go:29] interesting pod/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:23:20.527504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.527084    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:23:20.703875 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.703844    5817 generic.go:356] "Generic (PLEG): container finished" podID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerID="12e8e8a9aa645e5f247ecccdaffebbf68483ec8371a0641804d89531b5cd0493" exitCode=0
Oct 30 22:23:20.704044 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.703892    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" event={"ID":"05264fea-2fe9-4e27-9d1a-b667c43add57","Type":"ContainerDied","Data":"12e8e8a9aa645e5f247ecccdaffebbf68483ec8371a0641804d89531b5cd0493"}
Oct 30 22:23:20.704251 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.704236    5817 scope.go:122] "RemoveContainer" containerID="12e8e8a9aa645e5f247ecccdaffebbf68483ec8371a0641804d89531b5cd0493"
Oct 30 22:23:20.705504 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.705469    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:20.705599 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.705545    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:20.705599 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:20.705577    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:21.717848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:21.717809    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" event={"ID":"05264fea-2fe9-4e27-9d1a-b667c43add57","Type":"ContainerStarted","Data":"f8f4edc229277b9d16ae29869cd5f315483a19cbc8ac3abeeeaf3d8d27ccac22"}
Oct 30 22:23:21.964379 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:21.964338    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:23:21.964608 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:21.964583    5817 patch_prober.go:29] interesting pod/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:23:21.964676 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:21.964645    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:23:22.964532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:22.964495    5817 patch_prober.go:29] interesting pod/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:23:22.965112 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:22.964553    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:23:23.964928 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:23.964893    5817 patch_prober.go:29] interesting pod/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:23:23.965317 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:23.964954    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:23:24.965281 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:24.965248    5817 patch_prober.go:29] interesting pod/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:23:24.965697 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:24.965306    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:23:25.964670 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:25.964637    5817 patch_prober.go:29] interesting pod/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 22:23:25.964848 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:25.964704    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 22:23:26.968424 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:26.968395    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:23:26.968813 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:26.968723    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:23:26.969440 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:26.969421    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:23:39.011983 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:39.011900    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"]
Oct 30 22:23:39.121364 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:39.121316    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"]
Oct 30 22:23:44.012486 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.012427    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf" podUID="9863cecd-209e-420f-a311-028fc20d27b8" containerName="payload-processing" containerID="cri-o://63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2" gracePeriod=25
Oct 30 22:23:44.122543 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.122472    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9" podUID="5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" containerName="payload-pre-processing" containerID="cri-o://f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441" gracePeriod=25
Oct 30 22:23:44.269088 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.269018    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:23:44.359973 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.359936    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9863cecd-209e-420f-a311-028fc20d27b8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9863cecd-209e-420f-a311-028fc20d27b8-plugins-config-volume\") pod \"9863cecd-209e-420f-a311-028fc20d27b8\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") "
Oct 30 22:23:44.360129 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.360024    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9863cecd-209e-420f-a311-028fc20d27b8-kube-api-access-wxfwf\" (UniqueName: \"kubernetes.io/projected/9863cecd-209e-420f-a311-028fc20d27b8-kube-api-access-wxfwf\") pod \"9863cecd-209e-420f-a311-028fc20d27b8\" (UID: \"9863cecd-209e-420f-a311-028fc20d27b8\") "
Oct 30 22:23:44.360297 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.360273    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9863cecd-209e-420f-a311-028fc20d27b8-plugins-config-volume" pod "9863cecd-209e-420f-a311-028fc20d27b8" (UID: "9863cecd-209e-420f-a311-028fc20d27b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:44.361988 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.361967    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9863cecd-209e-420f-a311-028fc20d27b8-kube-api-access-wxfwf" pod "9863cecd-209e-420f-a311-028fc20d27b8" (UID: "9863cecd-209e-420f-a311-028fc20d27b8"). InnerVolumeSpecName "kube-api-access-wxfwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:44.372738 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.372717    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:23:44.461126 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.461097    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-kube-api-access-pnwvd\" (UniqueName: \"kubernetes.io/projected/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-kube-api-access-pnwvd\") pod \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") "
Oct 30 22:23:44.461274 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.461248    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-plugins-config-volume\") pod \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\" (UID: \"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5\") "
Oct 30 22:23:44.461622 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.461602    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9863cecd-209e-420f-a311-028fc20d27b8-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:44.461681 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.461627    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxfwf\" (UniqueName: \"kubernetes.io/projected/9863cecd-209e-420f-a311-028fc20d27b8-kube-api-access-wxfwf\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:44.461724 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.461712    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-plugins-config-volume" pod "5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" (UID: "5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:44.463068 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.463049    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-kube-api-access-pnwvd" pod "5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" (UID: "5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5"). InnerVolumeSpecName "kube-api-access-pnwvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:44.562607 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.562539    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:44.562607 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.562563    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnwvd\" (UniqueName: \"kubernetes.io/projected/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5-kube-api-access-pnwvd\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:44.833634 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.833553    5817 generic.go:356] "Generic (PLEG): container finished" podID="5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" containerID="f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441" exitCode=0
Oct 30 22:23:44.833634 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.833596    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"
Oct 30 22:23:44.833870 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.833641    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9" event={"ID":"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5","Type":"ContainerDied","Data":"f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441"}
Oct 30 22:23:44.833870 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.833698    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9" event={"ID":"5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5","Type":"ContainerDied","Data":"cff0a116c4dc80f51ca84f121bfff31488564647d2556541480450db4993531c"}
Oct 30 22:23:44.833870 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.833729    5817 scope.go:122] "RemoveContainer" containerID="f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441"
Oct 30 22:23:44.835087 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.835057    5817 generic.go:356] "Generic (PLEG): container finished" podID="9863cecd-209e-420f-a311-028fc20d27b8" containerID="63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2" exitCode=0
Oct 30 22:23:44.835185 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.835101    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"
Oct 30 22:23:44.835185 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.835106    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf" event={"ID":"9863cecd-209e-420f-a311-028fc20d27b8","Type":"ContainerDied","Data":"63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2"}
Oct 30 22:23:44.835185 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.835150    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf" event={"ID":"9863cecd-209e-420f-a311-028fc20d27b8","Type":"ContainerDied","Data":"fa1ad22f32b282a7f4eb9857826385ddefa15f5866032a03905309c106bea0bd"}
Oct 30 22:23:44.844756 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.844671    5817 scope.go:122] "RemoveContainer" containerID="f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441"
Oct 30 22:23:44.844977 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:23:44.844946    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441\": container with ID starting with f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441 not found: ID does not exist" containerID="f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441"
Oct 30 22:23:44.845096 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.844988    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441"} err="failed to get container status \"f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441\": rpc error: code = NotFound desc = could not find container \"f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441\": container with ID starting with f1ca6e4be0afe446d71ffaf533230a3933eef43a400f6c534ce283b9472f9441 not found: ID does not exist"
Oct 30 22:23:44.845096 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.845010    5817 scope.go:122] "RemoveContainer" containerID="63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2"
Oct 30 22:23:44.855193 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.855176    5817 scope.go:122] "RemoveContainer" containerID="63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2"
Oct 30 22:23:44.855482 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:23:44.855453    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2\": container with ID starting with 63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2 not found: ID does not exist" containerID="63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2"
Oct 30 22:23:44.855589 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.855488    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2"} err="failed to get container status \"63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2\": rpc error: code = NotFound desc = could not find container \"63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2\": container with ID starting with 63d09d8a85f10f4e184a4bcbd375e0b3077f4f39cace826ce33879e9ede0adc2 not found: ID does not exist"
Oct 30 22:23:44.995903 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:44.995873    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"]
Oct 30 22:23:45.042667 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:45.042638    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c501563-697bbbf86c-gfgc9"]
Oct 30 22:23:45.102995 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:45.102913    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"]
Oct 30 22:23:45.107547 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:45.107521    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c501563-7679965476-c46rf"]
Oct 30 22:23:46.337696 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:23:46.337663    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:23:46.841906 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:46.841872    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" path="/var/lib/kubelet/pods/5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5/volumes"
Oct 30 22:23:46.842523 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:46.842492    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9863cecd-209e-420f-a311-028fc20d27b8" path="/var/lib/kubelet/pods/9863cecd-209e-420f-a311-028fc20d27b8/volumes"
Oct 30 22:23:53.704411 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:53.704375    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"]
Oct 30 22:23:53.704799 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:53.704587    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" containerID="cri-o://f8f4edc229277b9d16ae29869cd5f315483a19cbc8ac3abeeeaf3d8d27ccac22" gracePeriod=30
Oct 30 22:23:55.609779 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.609746    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"]
Oct 30 22:23:55.610447 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610426    5817 cpu_manager.go:414] "containerMap: removing container" podUID="5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" containerName="payload-pre-processing"
Oct 30 22:23:55.610568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610447    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" containerName="payload-pre-processing"
Oct 30 22:23:55.610568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610499    5817 cpu_manager.go:414] "containerMap: removing container" podUID="9863cecd-209e-420f-a311-028fc20d27b8" containerName="payload-processing"
Oct 30 22:23:55.610568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610507    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9863cecd-209e-420f-a311-028fc20d27b8" containerName="payload-processing"
Oct 30 22:23:55.610568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610524    5817 cpu_manager.go:414] "containerMap: removing container" podUID="72d7fcdd-2942-4884-ac28-12fd93421de7" containerName="maas-api"
Oct 30 22:23:55.610568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610531    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72d7fcdd-2942-4884-ac28-12fd93421de7" containerName="maas-api"
Oct 30 22:23:55.610817 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610635    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="9863cecd-209e-420f-a311-028fc20d27b8" containerName="payload-processing"
Oct 30 22:23:55.610817 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610649    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="72d7fcdd-2942-4884-ac28-12fd93421de7" containerName="maas-api"
Oct 30 22:23:55.610817 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.610665    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b7d0704-ac63-4391-b1cc-d6b0eb7b00b5" containerName="payload-pre-processing"
Oct 30 22:23:55.614379 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.614335    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.664141 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.664107    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-8123f1-openshift-default-dockercfg-gm29j\""
Oct 30 22:23:55.676940 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.676913    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-socket\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677064 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.676955    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32054c4d-f019-4d2c-9100-efff95003da7-istiod-ca-cert\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677121 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.677052    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-certs\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677121 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.677104    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32054c4d-f019-4d2c-9100-efff95003da7-istio-podinfo\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677233 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.677209    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-data\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677301 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.677287    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-istio-token\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677365 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.677338    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6v8p\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-kube-api-access-z6v8p\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677433 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.677412    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-credential-socket\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.677487 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.677464    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-envoy\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.713090 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.713065    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"]
Oct 30 22:23:55.778667 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778616    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32054c4d-f019-4d2c-9100-efff95003da7-istio-podinfo\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.778849 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778679    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-data\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.778849 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778732    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-istio-token\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.778849 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778791    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6v8p\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-kube-api-access-z6v8p\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.778849 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778819    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-credential-socket\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.778849 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778846    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-envoy\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.779069 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778911    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-socket\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.779069 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778942    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32054c4d-f019-4d2c-9100-efff95003da7-istiod-ca-cert\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.779069 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.778996    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-certs\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.779490 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.779455    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-certs\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.779631 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.779601    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-data\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.779749 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.779716    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-credential-socket\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.781544 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.780811    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32054c4d-f019-4d2c-9100-efff95003da7-istiod-ca-cert\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.782083 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.781948    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-socket\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.785107 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.785075    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-envoy\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.785324 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.785301    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32054c4d-f019-4d2c-9100-efff95003da7-istio-podinfo\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.842699 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.842669    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-istio-token\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.842866 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.842848    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6v8p\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-kube-api-access-z6v8p\") pod \"e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") " pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:55.927434 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:55.927326    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:56.345402 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.345379    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"]
Oct 30 22:23:56.347977 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:23:56.347940    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32054c4d_f019_4d2c_9100_efff95003da7.slice/crio-608835b640edfd0b1d180dc0ac3edbe786e5f680e8c2088c11b4bb673bf25454 WatchSource:0}: Error finding container 608835b640edfd0b1d180dc0ac3edbe786e5f680e8c2088c11b4bb673bf25454: Status 404 returned error can't find the container with id 608835b640edfd0b1d180dc0ac3edbe786e5f680e8c2088c11b4bb673bf25454
Oct 30 22:23:56.350173 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.350140    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:56.350522 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.350205    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:56.350522 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.350232    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:23:56.906749 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.906716    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" event={"ID":"32054c4d-f019-4d2c-9100-efff95003da7","Type":"ContainerStarted","Data":"6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b"}
Oct 30 22:23:56.907130 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.906766    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" event={"ID":"32054c4d-f019-4d2c-9100-efff95003da7","Type":"ContainerStarted","Data":"608835b640edfd0b1d180dc0ac3edbe786e5f680e8c2088c11b4bb673bf25454"}
Oct 30 22:23:56.928042 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.928018    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:56.971398 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.971367    5817 patch_prober.go:29] interesting pod/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:23:56.971548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:56.971419    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:57.024548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:57.024489    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podStartSLOduration=2.024475856 podStartE2EDuration="2.024475856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:57.022540574 +0000 UTC m=+4172.310263121" watchObservedRunningTime="2026-10-30 22:23:57.024475856 +0000 UTC m=+4172.312198401"
Oct 30 22:23:57.933310 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:57.933275    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:58.918580 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:58.918547    5817 generic.go:356] "Generic (PLEG): container finished" podID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerID="f8f4edc229277b9d16ae29869cd5f315483a19cbc8ac3abeeeaf3d8d27ccac22" exitCode=0
Oct 30 22:23:58.918580 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:58.918574    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" event={"ID":"05264fea-2fe9-4e27-9d1a-b667c43add57","Type":"ContainerDied","Data":"f8f4edc229277b9d16ae29869cd5f315483a19cbc8ac3abeeeaf3d8d27ccac22"}
Oct 30 22:23:58.918857 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:58.918635    5817 scope.go:122] "RemoveContainer" containerID="12e8e8a9aa645e5f247ecccdaffebbf68483ec8371a0641804d89531b5cd0493"
Oct 30 22:23:58.919036 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:58.919019    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:58.920232 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:58.920217    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:23:58.957446 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:58.957421    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:23:59.012898 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.012820    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05264fea-2fe9-4e27-9d1a-b667c43add57-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05264fea-2fe9-4e27-9d1a-b667c43add57-istiod-ca-cert\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.012898 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.012863    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-kube-api-access-5ww29\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-kube-api-access-5ww29\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013105 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.012915    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-podinfo\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013105 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.012947    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-certs\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013105 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013032    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-token\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-token\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013105 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013074    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-data\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013312 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013147    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-envoy\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013312 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013217    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05264fea-2fe9-4e27-9d1a-b667c43add57-istiod-ca-cert" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:23:59.013312 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013230    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-credential-socket\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013530 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013376    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-certs" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:59.013530 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013475    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-socket\") pod \"05264fea-2fe9-4e27-9d1a-b667c43add57\" (UID: \"05264fea-2fe9-4e27-9d1a-b667c43add57\") "
Oct 30 22:23:59.013530 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013480    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-credential-socket" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:59.013762 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013740    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-socket" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:59.013857 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.013757    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-data" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:59.014140 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.014114    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.014292 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.014266    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.014402 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.014296    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05264fea-2fe9-4e27-9d1a-b667c43add57-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.014402 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.014311    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.014402 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.014327    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.015424 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.015403    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-kube-api-access-5ww29" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "kube-api-access-5ww29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:59.015523 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.015438    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-token" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:23:59.015651 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.015628    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-podinfo" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:23:59.015772 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.015752    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-envoy" pod "05264fea-2fe9-4e27-9d1a-b667c43add57" (UID: "05264fea-2fe9-4e27-9d1a-b667c43add57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:23:59.115965 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.115935    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ww29\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-kube-api-access-5ww29\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.115965 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.115958    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.115965 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.115968    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.116230 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.115977    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05264fea-2fe9-4e27-9d1a-b667c43add57-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:23:59.924607 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.924576    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"
Oct 30 22:23:59.924818 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.924618    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v" event={"ID":"05264fea-2fe9-4e27-9d1a-b667c43add57","Type":"ContainerDied","Data":"5880f7d5095e987aa1abec3a0967b2873cb57fd42c7d7f5beeb73b3c9a3bc420"}
Oct 30 22:23:59.924818 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:23:59.924674    5817 scope.go:122] "RemoveContainer" containerID="f8f4edc229277b9d16ae29869cd5f315483a19cbc8ac3abeeeaf3d8d27ccac22"
Oct 30 22:24:00.136261 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:00.136227    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"]
Oct 30 22:24:00.183089 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:00.183010    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8c501563-openshift-default-5bf7c8f47f-sk46v"]
Oct 30 22:24:00.841870 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:00.841835    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" path="/var/lib/kubelet/pods/05264fea-2fe9-4e27-9d1a-b667c43add57/volumes"
Oct 30 22:24:03.614576 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.614544    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"]
Oct 30 22:24:03.614986 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.614973    5817 cpu_manager.go:414] "containerMap: removing container" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy"
Oct 30 22:24:03.614986 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.614986    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy"
Oct 30 22:24:03.615065 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.615023    5817 cpu_manager.go:414] "containerMap: removing container" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy"
Oct 30 22:24:03.615065 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.615028    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy"
Oct 30 22:24:03.615133 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.615100    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy"
Oct 30 22:24:03.615133 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.615111    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="05264fea-2fe9-4e27-9d1a-b667c43add57" containerName="istio-proxy"
Oct 30 22:24:03.619611 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.619590    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.660714 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.660684    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8hx9\" (UniqueName: \"kubernetes.io/projected/545c0a00-3d10-4050-878e-4b32e63eeca7-kube-api-access-p8hx9\") pod \"payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.660881 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.660860    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/545c0a00-3d10-4050-878e-4b32e63eeca7-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.674726 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.674703    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"]
Oct 30 22:24:03.719225 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.719191    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-8123f1-dockercfg-qnrms\""
Oct 30 22:24:03.719453 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.719275    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-8123f1\""
Oct 30 22:24:03.754909 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.754873    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"]
Oct 30 22:24:03.758774 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.758751    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:03.762039 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.762013    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8hx9\" (UniqueName: \"kubernetes.io/projected/545c0a00-3d10-4050-878e-4b32e63eeca7-kube-api-access-p8hx9\") pod \"payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.762140 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.762116    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/545c0a00-3d10-4050-878e-4b32e63eeca7-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.762777 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.762754    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/545c0a00-3d10-4050-878e-4b32e63eeca7-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.813374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.813315    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"]
Oct 30 22:24:03.820796 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.820771    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8hx9\" (UniqueName: \"kubernetes.io/projected/545c0a00-3d10-4050-878e-4b32e63eeca7-kube-api-access-p8hx9\") pod \"payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.863629 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.863605    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsm5f\" (UniqueName: \"kubernetes.io/projected/275260ef-e5b3-45ed-9ad9-00dc5e1af237-kube-api-access-xsm5f\") pod \"payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:03.863751 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.863661    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/275260ef-e5b3-45ed-9ad9-00dc5e1af237-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:03.933043 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.932971    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:03.964643 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.964608    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/275260ef-e5b3-45ed-9ad9-00dc5e1af237-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:03.964900 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.964873    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsm5f\" (UniqueName: \"kubernetes.io/projected/275260ef-e5b3-45ed-9ad9-00dc5e1af237-kube-api-access-xsm5f\") pod \"payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:03.965523 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:03.965501    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/275260ef-e5b3-45ed-9ad9-00dc5e1af237-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:04.025947 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.025915    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsm5f\" (UniqueName: \"kubernetes.io/projected/275260ef-e5b3-45ed-9ad9-00dc5e1af237-kube-api-access-xsm5f\") pod \"payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:04.071805 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.071778    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:04.301532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.301502    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"]
Oct 30 22:24:04.304365 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:24:04.304320    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod545c0a00_3d10_4050_878e_4b32e63eeca7.slice/crio-e403549e91bc356a1405f373e8c42d5e4bf5553515635fd49f29b0bb4b3cb5e5 WatchSource:0}: Error finding container e403549e91bc356a1405f373e8c42d5e4bf5553515635fd49f29b0bb4b3cb5e5: Status 404 returned error can't find the container with id e403549e91bc356a1405f373e8c42d5e4bf5553515635fd49f29b0bb4b3cb5e5
Oct 30 22:24:04.439960 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.439936    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"]
Oct 30 22:24:04.442820 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:24:04.442787    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275260ef_e5b3_45ed_9ad9_00dc5e1af237.slice/crio-6e94098f1d0c26890fa2475b6844c4573039308ef7fee23a29223ed5c80922f2 WatchSource:0}: Error finding container 6e94098f1d0c26890fa2475b6844c4573039308ef7fee23a29223ed5c80922f2: Status 404 returned error can't find the container with id 6e94098f1d0c26890fa2475b6844c4573039308ef7fee23a29223ed5c80922f2
Oct 30 22:24:04.951855 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.951815    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6" event={"ID":"545c0a00-3d10-4050-878e-4b32e63eeca7","Type":"ContainerStarted","Data":"6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf"}
Oct 30 22:24:04.952296 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.951873    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6" event={"ID":"545c0a00-3d10-4050-878e-4b32e63eeca7","Type":"ContainerStarted","Data":"e403549e91bc356a1405f373e8c42d5e4bf5553515635fd49f29b0bb4b3cb5e5"}
Oct 30 22:24:04.952296 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.951904    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:04.953480 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.953433    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l" event={"ID":"275260ef-e5b3-45ed-9ad9-00dc5e1af237","Type":"ContainerStarted","Data":"060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583"}
Oct 30 22:24:04.953626 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.953497    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l" event={"ID":"275260ef-e5b3-45ed-9ad9-00dc5e1af237","Type":"ContainerStarted","Data":"6e94098f1d0c26890fa2475b6844c4573039308ef7fee23a29223ed5c80922f2"}
Oct 30 22:24:04.953626 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:04.953525    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:05.062835 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:05.062763    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6" podStartSLOduration=2.062748687 podStartE2EDuration="2.062748687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:05.061604509 +0000 UTC m=+4180.349327055" watchObservedRunningTime="2026-10-30 22:24:05.062748687 +0000 UTC m=+4180.350471232"
Oct 30 22:24:05.190822 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:05.190769    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l" podStartSLOduration=2.190755238 podStartE2EDuration="2.190755238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:05.189804882 +0000 UTC m=+4180.477527428" watchObservedRunningTime="2026-10-30 22:24:05.190755238 +0000 UTC m=+4180.478477784"
Oct 30 22:24:15.962125 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:15.962092    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:24:15.962596 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:15.962163    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:24:16.340209 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:24:16.340122    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:24:25.078719 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:25.078684    5817 generic.go:356] "Generic (PLEG): container finished" podID="32054c4d-f019-4d2c-9100-efff95003da7" containerID="6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b" exitCode=0
Oct 30 22:24:25.079169 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:25.078756    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" event={"ID":"32054c4d-f019-4d2c-9100-efff95003da7","Type":"ContainerDied","Data":"6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b"}
Oct 30 22:24:25.079169 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:25.079160    5817 scope.go:122] "RemoveContainer" containerID="6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b"
Oct 30 22:24:25.080452 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:25.080423    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:25.080520 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:25.080509    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:25.080556 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:25.080542    5817 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 22:24:26.085294 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:26.085262    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" event={"ID":"32054c4d-f019-4d2c-9100-efff95003da7","Type":"ContainerStarted","Data":"af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248"}
Oct 30 22:24:26.927908 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:26.927878    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:24:26.929125 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:26.929103    5817 patch_prober.go:29] interesting pod/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:24:26.929195 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:26.929151    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 22:24:27.928485 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:27.928452    5817 patch_prober.go:29] interesting pod/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:24:27.928911 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:27.928503    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 22:24:28.919623 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:28.919582    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:24:28.928233 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:28.928194    5817 patch_prober.go:29] interesting pod/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:24:28.928432 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:28.928259    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 22:24:29.928082 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:29.928049    5817 patch_prober.go:29] interesting pod/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:24:29.928658 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:29.928107    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 22:24:30.927798 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:30.927762    5817 patch_prober.go:29] interesting pod/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 22:24:30.927978 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:30.927822    5817 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 22:24:31.932616 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:31.932588    5817 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:24:31.933722 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:24:31.933699    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:24:46.337428 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:24:46.337389    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:25:16.338783 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:25:16.338704    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:25:46.336986 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:25:46.336945    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:25:57.161792 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:25:57.161757    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"]
Oct 30 22:25:57.269370 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:25:57.269322    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"]
Oct 30 22:26:02.162928 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.162870    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l" podUID="275260ef-e5b3-45ed-9ad9-00dc5e1af237" containerName="payload-processing" containerID="cri-o://060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583" gracePeriod=25
Oct 30 22:26:02.270459 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.270400    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6" podUID="545c0a00-3d10-4050-878e-4b32e63eeca7" containerName="payload-pre-processing" containerID="cri-o://6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf" gracePeriod=25
Oct 30 22:26:02.423021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.422959    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:26:02.527263 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.527239    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:26:02.588432 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.588393    5817 generic.go:356] "Generic (PLEG): container finished" podID="545c0a00-3d10-4050-878e-4b32e63eeca7" containerID="6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf" exitCode=0
Oct 30 22:26:02.588593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.588479    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6" event={"ID":"545c0a00-3d10-4050-878e-4b32e63eeca7","Type":"ContainerDied","Data":"6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf"}
Oct 30 22:26:02.588593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.588497    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"
Oct 30 22:26:02.588593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.588531    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6" event={"ID":"545c0a00-3d10-4050-878e-4b32e63eeca7","Type":"ContainerDied","Data":"e403549e91bc356a1405f373e8c42d5e4bf5553515635fd49f29b0bb4b3cb5e5"}
Oct 30 22:26:02.588593 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.588548    5817 scope.go:122] "RemoveContainer" containerID="6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf"
Oct 30 22:26:02.589951 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.589927    5817 generic.go:356] "Generic (PLEG): container finished" podID="275260ef-e5b3-45ed-9ad9-00dc5e1af237" containerID="060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583" exitCode=0
Oct 30 22:26:02.590036 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.589967    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l" event={"ID":"275260ef-e5b3-45ed-9ad9-00dc5e1af237","Type":"ContainerDied","Data":"060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583"}
Oct 30 22:26:02.590036 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.589970    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"
Oct 30 22:26:02.590036 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.590002    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l" event={"ID":"275260ef-e5b3-45ed-9ad9-00dc5e1af237","Type":"ContainerDied","Data":"6e94098f1d0c26890fa2475b6844c4573039308ef7fee23a29223ed5c80922f2"}
Oct 30 22:26:02.597239 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.597222    5817 scope.go:122] "RemoveContainer" containerID="6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf"
Oct 30 22:26:02.597522 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:26:02.597499    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf\": container with ID starting with 6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf not found: ID does not exist" containerID="6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf"
Oct 30 22:26:02.597627 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.597529    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf"} err="failed to get container status \"6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf\": rpc error: code = NotFound desc = could not find container \"6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf\": container with ID starting with 6a2c88810966ad8d4fab4dd33f92ef19bb70bcd1f63932a596488db460f844bf not found: ID does not exist"
Oct 30 22:26:02.597627 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.597544    5817 scope.go:122] "RemoveContainer" containerID="060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583"
Oct 30 22:26:02.605219 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.605203    5817 scope.go:122] "RemoveContainer" containerID="060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583"
Oct 30 22:26:02.605463 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.605441    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/275260ef-e5b3-45ed-9ad9-00dc5e1af237-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/275260ef-e5b3-45ed-9ad9-00dc5e1af237-plugins-config-volume\") pod \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") "
Oct 30 22:26:02.605532 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:26:02.605475    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583\": container with ID starting with 060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583 not found: ID does not exist" containerID="060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583"
Oct 30 22:26:02.605532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.605498    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583"} err="failed to get container status \"060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583\": rpc error: code = NotFound desc = could not find container \"060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583\": container with ID starting with 060a796ffda5480a71a17bfcbf15edcdd65936781d5239db1551b8ee27c3d583 not found: ID does not exist"
Oct 30 22:26:02.605532 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.605479    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/275260ef-e5b3-45ed-9ad9-00dc5e1af237-kube-api-access-xsm5f\" (UniqueName: \"kubernetes.io/projected/275260ef-e5b3-45ed-9ad9-00dc5e1af237-kube-api-access-xsm5f\") pod \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\" (UID: \"275260ef-e5b3-45ed-9ad9-00dc5e1af237\") "
Oct 30 22:26:02.605794 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.605775    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/275260ef-e5b3-45ed-9ad9-00dc5e1af237-plugins-config-volume" pod "275260ef-e5b3-45ed-9ad9-00dc5e1af237" (UID: "275260ef-e5b3-45ed-9ad9-00dc5e1af237"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:02.605942 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.605928    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/275260ef-e5b3-45ed-9ad9-00dc5e1af237-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:02.607354 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.607324    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/275260ef-e5b3-45ed-9ad9-00dc5e1af237-kube-api-access-xsm5f" pod "275260ef-e5b3-45ed-9ad9-00dc5e1af237" (UID: "275260ef-e5b3-45ed-9ad9-00dc5e1af237"). InnerVolumeSpecName "kube-api-access-xsm5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:02.707092 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.707070    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/545c0a00-3d10-4050-878e-4b32e63eeca7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/545c0a00-3d10-4050-878e-4b32e63eeca7-plugins-config-volume\") pod \"545c0a00-3d10-4050-878e-4b32e63eeca7\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") "
Oct 30 22:26:02.707238 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.707135    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/545c0a00-3d10-4050-878e-4b32e63eeca7-kube-api-access-p8hx9\" (UniqueName: \"kubernetes.io/projected/545c0a00-3d10-4050-878e-4b32e63eeca7-kube-api-access-p8hx9\") pod \"545c0a00-3d10-4050-878e-4b32e63eeca7\" (UID: \"545c0a00-3d10-4050-878e-4b32e63eeca7\") "
Oct 30 22:26:02.707508 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.707486    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsm5f\" (UniqueName: \"kubernetes.io/projected/275260ef-e5b3-45ed-9ad9-00dc5e1af237-kube-api-access-xsm5f\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:02.707592 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.707484    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/545c0a00-3d10-4050-878e-4b32e63eeca7-plugins-config-volume" pod "545c0a00-3d10-4050-878e-4b32e63eeca7" (UID: "545c0a00-3d10-4050-878e-4b32e63eeca7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:02.709112 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.709091    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/545c0a00-3d10-4050-878e-4b32e63eeca7-kube-api-access-p8hx9" pod "545c0a00-3d10-4050-878e-4b32e63eeca7" (UID: "545c0a00-3d10-4050-878e-4b32e63eeca7"). InnerVolumeSpecName "kube-api-access-p8hx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:02.808229 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.808205    5817 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/545c0a00-3d10-4050-878e-4b32e63eeca7-plugins-config-volume\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:02.808229 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:02.808227    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8hx9\" (UniqueName: \"kubernetes.io/projected/545c0a00-3d10-4050-878e-4b32e63eeca7-kube-api-access-p8hx9\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:03.000548 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:03.000456    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"]
Oct 30 22:26:03.002079 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:03.002054    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-8123f1-6f77746f5f-hvzf6"]
Oct 30 22:26:03.105545 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:03.105514    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"]
Oct 30 22:26:03.110618 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:03.110592    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-8123f1-c7bd97654-vv79l"]
Oct 30 22:26:04.842282 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:04.842246    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="275260ef-e5b3-45ed-9ad9-00dc5e1af237" path="/var/lib/kubelet/pods/275260ef-e5b3-45ed-9ad9-00dc5e1af237/volumes"
Oct 30 22:26:04.842828 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:04.842805    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="545c0a00-3d10-4050-878e-4b32e63eeca7" path="/var/lib/kubelet/pods/545c0a00-3d10-4050-878e-4b32e63eeca7/volumes"
Oct 30 22:26:16.244160 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:16.244124    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"]
Oct 30 22:26:16.244637 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:16.244325    5817 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy" containerID="cri-o://af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248" gracePeriod=30
Oct 30 22:26:16.349519 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:26:16.349473    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:26:16.935738 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:16.935700    5817 patch_prober.go:29] interesting pod/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:26:16.935909 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:16.935765    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:26:21.503554 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.503527    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:26:21.600206 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600139    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-kube-api-access-z6v8p\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-kube-api-access-z6v8p\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600206 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600179    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-credential-socket\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600450 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600215    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-istio-token\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-istio-token\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600450 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600240    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-certs\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600450 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600278    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-socket\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600450 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600309    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/32054c4d-f019-4d2c-9100-efff95003da7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32054c4d-f019-4d2c-9100-efff95003da7-istio-podinfo\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600450 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600341    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-envoy\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600450 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600404    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32054c4d-f019-4d2c-9100-efff95003da7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32054c4d-f019-4d2c-9100-efff95003da7-istiod-ca-cert\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600450 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600427    5817 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-data\") pod \"32054c4d-f019-4d2c-9100-efff95003da7\" (UID: \"32054c4d-f019-4d2c-9100-efff95003da7\") "
Oct 30 22:26:21.600825 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600467    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-credential-socket" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:21.600888 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600862    5817 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-credential-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.600949 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600898    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-socket" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:21.601021 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.600955    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-certs" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:21.601073 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.601048    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32054c4d-f019-4d2c-9100-efff95003da7-istiod-ca-cert" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:21.601073 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.601058    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-data" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:21.602621 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.602587    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32054c4d-f019-4d2c-9100-efff95003da7-istio-podinfo" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:26:21.602732 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.602685    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-istio-token" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:21.602849 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.602828    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-envoy" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:21.602885 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.602850    5817 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-kube-api-access-z6v8p" pod "32054c4d-f019-4d2c-9100-efff95003da7" (UID: "32054c4d-f019-4d2c-9100-efff95003da7"). InnerVolumeSpecName "kube-api-access-z6v8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:21.686279 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.686243    5817 generic.go:356] "Generic (PLEG): container finished" podID="32054c4d-f019-4d2c-9100-efff95003da7" containerID="af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248" exitCode=0
Oct 30 22:26:21.686451 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.686297    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" event={"ID":"32054c4d-f019-4d2c-9100-efff95003da7","Type":"ContainerDied","Data":"af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248"}
Oct 30 22:26:21.686451 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.686308    5817 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"
Oct 30 22:26:21.686451 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.686401    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f" event={"ID":"32054c4d-f019-4d2c-9100-efff95003da7","Type":"ContainerDied","Data":"608835b640edfd0b1d180dc0ac3edbe786e5f680e8c2088c11b4bb673bf25454"}
Oct 30 22:26:21.686451 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.686434    5817 scope.go:122] "RemoveContainer" containerID="af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248"
Oct 30 22:26:21.698856 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.698838    5817 scope.go:122] "RemoveContainer" containerID="6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b"
Oct 30 22:26:21.701492 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701471    5817 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6v8p\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-kube-api-access-z6v8p\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.701492 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701497    5817 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32054c4d-f019-4d2c-9100-efff95003da7-istio-token\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.701645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701514    5817 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-certs\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.701645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701527    5817 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-workload-socket\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.701645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701542    5817 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32054c4d-f019-4d2c-9100-efff95003da7-istio-podinfo\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.701645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701554    5817 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-envoy\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.701645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701568    5817 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32054c4d-f019-4d2c-9100-efff95003da7-istiod-ca-cert\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.701645 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.701581    5817 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32054c4d-f019-4d2c-9100-efff95003da7-istio-data\") on node \"ip-10-0-142-25.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:21.707292 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.707268    5817 scope.go:122] "RemoveContainer" containerID="af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248"
Oct 30 22:26:21.707569 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:26:21.707549    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248\": container with ID starting with af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248 not found: ID does not exist" containerID="af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248"
Oct 30 22:26:21.707696 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.707577    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248"} err="failed to get container status \"af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248\": rpc error: code = NotFound desc = could not find container \"af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248\": container with ID starting with af3bf25a975ea4738095391978f9a95837f83a8f8e9711efa458f19436618248 not found: ID does not exist"
Oct 30 22:26:21.707696 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.707597    5817 scope.go:122] "RemoveContainer" containerID="6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b"
Oct 30 22:26:21.707892 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:26:21.707874    5817 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b\": container with ID starting with 6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b not found: ID does not exist" containerID="6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b"
Oct 30 22:26:21.707933 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.707898    5817 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b"} err="failed to get container status \"6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b\": rpc error: code = NotFound desc = could not find container \"6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b\": container with ID starting with 6733e48448c3aa32961b2e97c79ed06af7c0b58f0906459f8f791c202c7b270b not found: ID does not exist"
Oct 30 22:26:21.843457 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.843426    5817 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"]
Oct 30 22:26:21.847908 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:21.847887    5817 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-8123f1-openshift-default-f5cfcfbd5-7j74f"]
Oct 30 22:26:22.841924 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:26:22.841892    5817 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32054c4d-f019-4d2c-9100-efff95003da7" path="/var/lib/kubelet/pods/32054c4d-f019-4d2c-9100-efff95003da7/volumes"
Oct 30 22:26:46.340712 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:26:46.340633    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:27:16.338409 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:27:16.338339    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:27:27.431230 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:27.431198    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86978f68d-xztz5_efa553c6-4d31-47f8-8baf-d38186d20bae/authorino/0.log"
Oct 30 22:27:30.164374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:30.162968    5817 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-67bf986f99-tr966 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:27:30.164374 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:30.163049    5817 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-67bf986f99-tr966" podUID="be00a1a8-0b75-48fa-8863-c2150f20d975" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:27:38.673110 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:38.673078    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-69c889db49-4gj8j_13f4baa5-f7c2-45ff-955e-60c8722193d0/manager/0.log"
Oct 30 22:27:40.184403 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:40.184373    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-744f599cc5-9q2r7_6767a5f0-120f-4614-9fb0-4b251243b773/manager/0.log"
Oct 30 22:27:41.981770 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:41.981741    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6687d569c9-6vwg5_4b8b92c4-8372-4f36-951e-31db7f96cc47/maas-api/0.log"
Oct 30 22:27:44.665601 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:44.665562    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86978f68d-xztz5_efa553c6-4d31-47f8-8baf-d38186d20bae/authorino/0.log"
Oct 30 22:27:46.337562 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:27:46.337526    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:27:47.884445 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:47.884417    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6645f5c86b-9sbgz_1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04/kube-auth-proxy/0.log"
Oct 30 22:27:48.582635 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:48.582604    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-hkq67_a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878/istio-proxy/0.log"
Oct 30 22:27:49.131901 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:49.131864    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-jsclw_7d34de55-a99d-4f02-b45e-915395b7d0b4/istio-proxy/0.log"
Oct 30 22:27:49.866394 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:49.866358    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-s22vv_9c1c929f-9bf9-43f3-bf99-3c0d913b1913/istio-proxy/0.log"
Oct 30 22:27:50.344272 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:50.344243    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-tr966_be00a1a8-0b75-48fa-8863-c2150f20d975/istio-proxy/0.log"
Oct 30 22:27:53.427070 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:53.427040    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-t8jcx_5281654a-072a-4d7d-bce5-198d3047aa40/main/0.log"
Oct 30 22:27:53.689235 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:53.689162    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r_e8da95b8-bbb8-4d66-b355-4d99d050bc64/main/0.log"
Oct 30 22:27:54.129043 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:54.129016    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v_41f13ead-4edc-44a9-a810-8eb47a1b51f2/main/0.log"
Oct 30 22:27:55.321367 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:27:55.321326    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86978f68d-xztz5_efa553c6-4d31-47f8-8baf-d38186d20bae/authorino/0.log"
Oct 30 22:28:04.838432 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:04.838314    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-69c889db49-4gj8j_13f4baa5-f7c2-45ff-955e-60c8722193d0/manager/0.log"
Oct 30 22:28:06.261799 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:06.261757    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-744f599cc5-9q2r7_6767a5f0-120f-4614-9fb0-4b251243b773/manager/0.log"
Oct 30 22:28:07.855126 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:07.855089    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6687d569c9-6vwg5_4b8b92c4-8372-4f36-951e-31db7f96cc47/maas-api/0.log"
Oct 30 22:28:10.371902 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:10.371873    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86978f68d-xztz5_efa553c6-4d31-47f8-8baf-d38186d20bae/authorino/0.log"
Oct 30 22:28:13.718104 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:13.718069    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6645f5c86b-9sbgz_1e74b7b9-a7cd-4eaf-ba8d-da7805d0ed04/kube-auth-proxy/0.log"
Oct 30 22:28:14.483666 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:14.483635    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-hkq67_a32c9c9e-9f2c-4b26-ad2f-7f81b04fd878/istio-proxy/0.log"
Oct 30 22:28:15.163441 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:15.163410    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-jsclw_7d34de55-a99d-4f02-b45e-915395b7d0b4/istio-proxy/0.log"
Oct 30 22:28:16.095099 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:16.095069    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-s22vv_9c1c929f-9bf9-43f3-bf99-3c0d913b1913/istio-proxy/0.log"
Oct 30 22:28:16.336835 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:28:16.336801    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:28:16.683497 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:16.683463    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-67bf986f99-tr966_be00a1a8-0b75-48fa-8863-c2150f20d975/istio-proxy/0.log"
Oct 30 22:28:19.505790 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:19.505763    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-t8jcx_5281654a-072a-4d7d-bce5-198d3047aa40/main/0.log"
Oct 30 22:28:19.786333 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:19.786257    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dvh7r_e8da95b8-bbb8-4d66-b355-4d99d050bc64/main/0.log"
Oct 30 22:28:20.304810 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:20.304778    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-zjr7v_41f13ead-4edc-44a9-a810-8eb47a1b51f2/main/0.log"
Oct 30 22:28:46.337740 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:28:46.337701    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:28:49.860757 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:49.860729    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-744f599cc5-9q2r7_6767a5f0-120f-4614-9fb0-4b251243b773/manager/0.log"
Oct 30 22:28:50.424356 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:28:50.424316    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-744f599cc5-9q2r7_6767a5f0-120f-4614-9fb0-4b251243b773/manager/0.log"
Oct 30 22:29:16.337738 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:29:16.337698    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:29:31.145160 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:31.145126    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-744f599cc5-9q2r7_6767a5f0-120f-4614-9fb0-4b251243b773/manager/0.log"
Oct 30 22:29:31.685850 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:31.685823    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-744f599cc5-9q2r7_6767a5f0-120f-4614-9fb0-4b251243b773/manager/0.log"
Oct 30 22:29:43.603568 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:43.603491    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-26g9z_2e048855-3db9-47b4-a1dc-29ee474f05e0/global-pull-secret-syncer/0.log"
Oct 30 22:29:46.337659 ip-10-0-142-25 kubenswrapper[5817]: E1030 22:29:46.337614    5817 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-8vhcn" containerName="extractor"
Oct 30 22:29:47.279924 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:47.279894    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d4w2x_ce211496-a484-4fc1-bf17-3a926847d855/konnectivity-agent/0.log"
Oct 30 22:29:49.162543 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:49.162514    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-25.us-west-2.compute.internal_30e0ff2135e8ccc515bab6d6f7a12d83/haproxy/0.log"
Oct 30 22:29:51.197831 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:51.197795    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-86978f68d-xztz5_efa553c6-4d31-47f8-8baf-d38186d20bae/authorino/0.log"
Oct 30 22:29:52.705557 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.705523    5817 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"]
Oct 30 22:29:52.705947 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.705934    5817 cpu_manager.go:414] "containerMap: removing container" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy"
Oct 30 22:29:52.705947 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.705945    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy"
Oct 30 22:29:52.706022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.705976    5817 cpu_manager.go:414] "containerMap: removing container" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy"
Oct 30 22:29:52.706022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.705982    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy"
Oct 30 22:29:52.706022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.705996    5817 cpu_manager.go:414] "containerMap: removing container" podUID="275260ef-e5b3-45ed-9ad9-00dc5e1af237" containerName="payload-processing"
Oct 30 22:29:52.706022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.706003    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="275260ef-e5b3-45ed-9ad9-00dc5e1af237" containerName="payload-processing"
Oct 30 22:29:52.706022 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.706018    5817 cpu_manager.go:414] "containerMap: removing container" podUID="545c0a00-3d10-4050-878e-4b32e63eeca7" containerName="payload-pre-processing"
Oct 30 22:29:52.706179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.706023    5817 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="545c0a00-3d10-4050-878e-4b32e63eeca7" containerName="payload-pre-processing"
Oct 30 22:29:52.706179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.706083    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="545c0a00-3d10-4050-878e-4b32e63eeca7" containerName="payload-pre-processing"
Oct 30 22:29:52.706179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.706090    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="32054c4d-f019-4d2c-9100-efff95003da7" containerName="istio-proxy"
Oct 30 22:29:52.706179 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.706104    5817 memory_manager.go:365] "RemoveStaleState removing state" podUID="275260ef-e5b3-45ed-9ad9-00dc5e1af237" containerName="payload-processing"
Oct 30 22:29:52.709064 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.709041    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.759605 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.759570    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hgcw8\"/\"openshift-service-ca.crt\""
Oct 30 22:29:52.795650 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.795612    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-proc\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.795842 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.795668    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-podres\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.795842 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.795779    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj6cn\" (UniqueName: \"kubernetes.io/projected/38049446-376b-41aa-ae5a-d7c83ae015ce-kube-api-access-fj6cn\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.795992 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.795969    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-sys\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.796050 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.796029    5817 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-lib-modules\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.808149 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.808116    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"]
Oct 30 22:29:52.854623 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.854591    5817 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hgcw8\"/\"kube-root-ca.crt\""
Oct 30 22:29:52.854803 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.854704    5817 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hgcw8\"/\"default-dockercfg-sqfbc\""
Oct 30 22:29:52.897355 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897311    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-podres\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.897538 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897380    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj6cn\" (UniqueName: \"kubernetes.io/projected/38049446-376b-41aa-ae5a-d7c83ae015ce-kube-api-access-fj6cn\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.897538 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897438    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-podres\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.897673 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897586    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-sys\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.897673 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897653    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-lib-modules\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.897800 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897729    5817 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-proc\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.897956 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897930    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-sys\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.898029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.897986    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-proc\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.898029 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.898011    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38049446-376b-41aa-ae5a-d7c83ae015ce-lib-modules\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:52.955337 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:52.955300    5817 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj6cn\" (UniqueName: \"kubernetes.io/projected/38049446-376b-41aa-ae5a-d7c83ae015ce-kube-api-access-fj6cn\") pod \"perf-node-gather-daemonset-n5nbb\" (UID: \"38049446-376b-41aa-ae5a-d7c83ae015ce\") " pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:53.024007 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:53.023936    5817 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:53.393439 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:53.393410    5817 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"]
Oct 30 22:29:53.395796 ip-10-0-142-25 kubenswrapper[5817]: W1030 22:29:53.395765    5817 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod38049446_376b_41aa_ae5a_d7c83ae015ce.slice/crio-12e53e4b1065d36c33b232119e9e5f74613debcf8a5f08a08f9b321139814457 WatchSource:0}: Error finding container 12e53e4b1065d36c33b232119e9e5f74613debcf8a5f08a08f9b321139814457: Status 404 returned error can't find the container with id 12e53e4b1065d36c33b232119e9e5f74613debcf8a5f08a08f9b321139814457
Oct 30 22:29:53.871326 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:53.871292    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb" event={"ID":"38049446-376b-41aa-ae5a-d7c83ae015ce","Type":"ContainerStarted","Data":"919d31b9c212d09cf902dd39c1df40cfcae6b2fc79438557042776119c4902f9"}
Oct 30 22:29:53.871747 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:53.871366    5817 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb" event={"ID":"38049446-376b-41aa-ae5a-d7c83ae015ce","Type":"ContainerStarted","Data":"12e53e4b1065d36c33b232119e9e5f74613debcf8a5f08a08f9b321139814457"}
Oct 30 22:29:53.871747 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:53.871396    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:29:53.983253 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:53.983198    5817 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb" podStartSLOduration=1.983185387 podStartE2EDuration="1.983185387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:53.981278237 +0000 UTC m=+4529.269000783" watchObservedRunningTime="2026-10-30 22:29:53.983185387 +0000 UTC m=+4529.270907933"
Oct 30 22:29:56.118897 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:56.118872    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1acc732-57dc-4f89-9880-7680cb0d9471/alertmanager/0.log"
Oct 30 22:29:56.517720 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:56.517694    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1acc732-57dc-4f89-9880-7680cb0d9471/config-reloader/0.log"
Oct 30 22:29:56.920415 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:56.920309    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1acc732-57dc-4f89-9880-7680cb0d9471/kube-rbac-proxy-web/0.log"
Oct 30 22:29:57.315363 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:57.315316    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1acc732-57dc-4f89-9880-7680cb0d9471/kube-rbac-proxy/0.log"
Oct 30 22:29:57.711034 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:57.711010    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1acc732-57dc-4f89-9880-7680cb0d9471/kube-rbac-proxy-metric/0.log"
Oct 30 22:29:58.110049 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:58.109970    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1acc732-57dc-4f89-9880-7680cb0d9471/prom-label-proxy/0.log"
Oct 30 22:29:58.536680 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:58.536648    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b1acc732-57dc-4f89-9880-7680cb0d9471/init-config-reloader/0.log"
Oct 30 22:29:59.314426 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:59.314396    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-rr7fk_a99b0768-fe06-4d64-95cb-64f5334596f8/kube-state-metrics/0.log"
Oct 30 22:29:59.710398 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:59.710339    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-rr7fk_a99b0768-fe06-4d64-95cb-64f5334596f8/kube-rbac-proxy-main/0.log"
Oct 30 22:29:59.887147 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:29:59.887123    5817 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hgcw8/perf-node-gather-daemonset-n5nbb"
Oct 30 22:30:00.105516 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:00.105450    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-rr7fk_a99b0768-fe06-4d64-95cb-64f5334596f8/kube-rbac-proxy-self/0.log"
Oct 30 22:30:00.923024 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:00.922991    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-gfbx5_85b75f40-bfd6-4ed1-88e8-0c56ee7d41f7/monitoring-plugin/0.log"
Oct 30 22:30:03.864098 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:03.864052    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t47rj_807ebbbd-72a2-47c3-b50a-2b044f71a8ff/node-exporter/0.log"
Oct 30 22:30:04.283720 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:04.283688    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t47rj_807ebbbd-72a2-47c3-b50a-2b044f71a8ff/kube-rbac-proxy/0.log"
Oct 30 22:30:04.682709 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:04.682641    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t47rj_807ebbbd-72a2-47c3-b50a-2b044f71a8ff/init-textfile/0.log"
Oct 30 22:30:05.084479 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:05.084445    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-vcnbh_edbff937-2134-4112-8ff4-3c50ed16ffe2/kube-rbac-proxy-main/0.log"
Oct 30 22:30:05.481738 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:05.481710    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-vcnbh_edbff937-2134-4112-8ff4-3c50ed16ffe2/kube-rbac-proxy-self/0.log"
Oct 30 22:30:05.878235 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:05.878164    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-vcnbh_edbff937-2134-4112-8ff4-3c50ed16ffe2/openshift-state-metrics/0.log"
Oct 30 22:30:06.302621 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:06.302589    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e02771be-80a2-4d57-bccf-c401aa52ca2a/prometheus/0.log"
Oct 30 22:30:06.706832 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:06.706802    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e02771be-80a2-4d57-bccf-c401aa52ca2a/config-reloader/0.log"
Oct 30 22:30:07.103876 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:07.103795    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e02771be-80a2-4d57-bccf-c401aa52ca2a/thanos-sidecar/0.log"
Oct 30 22:30:07.498614 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:07.498577    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e02771be-80a2-4d57-bccf-c401aa52ca2a/kube-rbac-proxy-web/0.log"
Oct 30 22:30:07.965184 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:07.965156    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e02771be-80a2-4d57-bccf-c401aa52ca2a/kube-rbac-proxy/0.log"
Oct 30 22:30:08.366236 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:08.366166    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e02771be-80a2-4d57-bccf-c401aa52ca2a/kube-rbac-proxy-thanos/0.log"
Oct 30 22:30:08.762079 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:08.762051    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e02771be-80a2-4d57-bccf-c401aa52ca2a/init-config-reloader/0.log"
Oct 30 22:30:09.171200 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:09.171132    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-nkmlv_ff1ba61a-eb5d-4b06-8690-b564a17d47fd/prometheus-operator/0.log"
Oct 30 22:30:09.563479 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:09.563443    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-nkmlv_ff1ba61a-eb5d-4b06-8690-b564a17d47fd/kube-rbac-proxy/0.log"
Oct 30 22:30:09.963628 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:09.963588    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-l4g59_49b86000-eb20-4c99-9bfe-7269ea162576/prometheus-operator-admission-webhook/0.log"
Oct 30 22:30:10.380015 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:10.379936    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567d7b57f6-qctkw_3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2/telemeter-client/0.log"
Oct 30 22:30:10.775950 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:10.775922    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567d7b57f6-qctkw_3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2/reload/0.log"
Oct 30 22:30:11.179511 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:11.179429    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-567d7b57f6-qctkw_3fe93ae4-ea86-46a8-8d23-e85c5dc3f2e2/kube-rbac-proxy/0.log"
Oct 30 22:30:11.584202 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:11.584173    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c84f565c4-h97ps_0bacc5cd-6d42-48f6-8807-5117835ca615/thanos-query/0.log"
Oct 30 22:30:11.996200 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:11.996169    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c84f565c4-h97ps_0bacc5cd-6d42-48f6-8807-5117835ca615/kube-rbac-proxy-web/0.log"
Oct 30 22:30:12.393573 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:12.393506    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c84f565c4-h97ps_0bacc5cd-6d42-48f6-8807-5117835ca615/kube-rbac-proxy/0.log"
Oct 30 22:30:12.795245 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:12.795216    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c84f565c4-h97ps_0bacc5cd-6d42-48f6-8807-5117835ca615/prom-label-proxy/0.log"
Oct 30 22:30:13.191901 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:13.191819    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c84f565c4-h97ps_0bacc5cd-6d42-48f6-8807-5117835ca615/kube-rbac-proxy-rules/0.log"
Oct 30 22:30:13.618813 ip-10-0-142-25 kubenswrapper[5817]: I1030 22:30:13.618782    5817 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c84f565c4-h97ps_0bacc5cd-6d42-48f6-8807-5117835ca615/kube-rbac-proxy-metrics/0.log"