Oct 31 07:28:27.755732 ip-10-0-133-50 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 07:28:28.153529 ip-10-0-133-50 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:28:28.153529 ip-10-0-133-50 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 07:28:28.153529 ip-10-0-133-50 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:28:28.153529 ip-10-0-133-50 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:28:28.158178 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158099    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.158178 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158174    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.158178 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158179    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.158178 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158183    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158187    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158192    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158195    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158199    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158204    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158207    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158211    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158215    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158219    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158223    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158228    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158232    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158236    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158240    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158244    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158248    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158251    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158255    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.158433 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158259    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158263    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158267    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158271    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158275    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158278    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158282    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158287    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158290    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158294    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158299    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158303    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158308    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158311    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158316    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158321    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158326    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158330    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158334    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158338    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.159211 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158342    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158347    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158352    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158357    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158361    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158366    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158370    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158374    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158378    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158383    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158387    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158392    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158396    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158401    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158405    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158410    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158414    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158418    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158423    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.159709 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158427    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158432    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158436    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158440    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158445    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158449    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158453    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158457    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158485    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158491    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158496    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158502    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158506    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158510    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158515    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158519    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158524    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158528    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158533    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158537    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.160336 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158541    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158546    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158550    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158556    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158563    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158568    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158573    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158578    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158582    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158587    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158591    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158596    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158600    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158605    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158609    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158615    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158619    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158623    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158628    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.161208 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158632    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158637    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158642    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158656    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158661    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.158666    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160712    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160724    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160729    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160736    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160743    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160748    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160754    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160759    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160763    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160767    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160772    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160776    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160780    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.161933 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160784    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160789    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160793    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160797    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160801    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160806    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160810    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160814    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160819    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160823    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160828    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160832    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160837    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160841    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160846    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160851    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160855    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160860    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160864    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160869    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.162403 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160874    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160881    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160886    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160891    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160897    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160901    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160905    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160909    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160914    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160918    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160922    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160926    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160931    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160935    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160939    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160944    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160948    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160952    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160957    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.163181 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160962    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160966    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160971    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160975    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160979    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160984    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160988    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160992    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.160997    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161002    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161006    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161011    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161015    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161020    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161024    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161028    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161032    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161037    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161041    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161045    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.163804 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161049    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161053    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161057    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161061    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161065    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161070    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161075    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161079    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161084    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161088    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161092    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161097    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161101    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161106    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161110    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161114    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161118    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161122    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161126    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161131    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.164277 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161136    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161140    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161144    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161148    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161152    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161156    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161163    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161168    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161172    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161176    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161180    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161184    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161189    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.161193    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162841    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162853    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162858    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162865    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162872    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.165117 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162877    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162882    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162886    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162891    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162895    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162899    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162903    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162908    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162912    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162917    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162921    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162925    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162930    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162934    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162939    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162943    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162947    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162952    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162956    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162964    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.165939 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162969    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162973    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162977    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162981    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162985    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162989    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162994    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.162998    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163002    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163007    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163011    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163016    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163020    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163024    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163028    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163032    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163037    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163042    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163046    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163050    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.166802 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163054    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163058    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163062    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163067    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163071    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163075    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163079    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163083    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163087    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163091    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163095    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163100    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163106    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163110    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163114    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163119    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163123    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163127    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163131    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.167437 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163139    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163144    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163149    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163154    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163158    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163162    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163166    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163171    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163175    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163181    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163185    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163189    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163193    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163197    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163201    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163205    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163215    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163220    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163225    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163229    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.168001 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163233    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163237    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163241    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163246    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163250    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163253    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163259    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163263    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163270    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163274    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163278    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163282    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163286    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163290    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163294    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163299    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163303    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163308    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163312    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.168533 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163316    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163320    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.163328    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164367    2577 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164382    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164393    2577 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164399    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164407    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164413    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164420    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164426    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164431    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164437    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164442    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164447    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164452    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164457    2577 flags.go:64] FLAG: --cgroup-root=""
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164478    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164484    2577 flags.go:64] FLAG: --client-ca-file=""
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164488    2577 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164494    2577 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164501    2577 flags.go:64] FLAG: --cluster-domain=""
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164505    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164510    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 07:28:28.169017 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164515    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164521    2577 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164527    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164532    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164537    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164543    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164548    2577 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164552    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164557    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164562    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164568    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164574    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164580    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164584    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164589    2577 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164594    2577 flags.go:64] FLAG: --enable-server="true"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164598    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164606    2577 flags.go:64] FLAG: --event-burst="100"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164611    2577 flags.go:64] FLAG: --event-qps="50"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164616    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164621    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164626    2577 flags.go:64] FLAG: --eviction-hard=""
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164632    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164637    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164642    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 07:28:28.169647 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164647    2577 flags.go:64] FLAG: --eviction-soft=""
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164652    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164656    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164661    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164666    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164671    2577 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164676    2577 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164680    2577 flags.go:64] FLAG: --feature-gates=""
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164687    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164692    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164697    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164702    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164706    2577 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164711    2577 flags.go:64] FLAG: --help="false"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164716    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164721    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164726    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164731    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164736    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164742    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164747    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164751    2577 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164756    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 07:28:28.170272 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164760    2577 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164765    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164770    2577 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164774    2577 flags.go:64] FLAG: --kube-reserved=""
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164780    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164784    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164790    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164794    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164799    2577 flags.go:64] FLAG: --lock-file=""
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164803    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164808    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164813    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164822    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164826    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164840    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164845    2577 flags.go:64] FLAG: --logging-format="text"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164850    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164855    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164860    2577 flags.go:64] FLAG: --manifest-url=""
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164865    2577 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164872    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164877    2577 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164883    2577 flags.go:64] FLAG: --max-pods="110"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164888    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164893    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 07:28:28.170829 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164897    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164902    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164907    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164912    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164917    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164927    2577 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164932    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164938    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164943    2577 flags.go:64] FLAG: --pod-cidr=""
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164947    2577 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164952    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164957    2577 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164962    2577 flags.go:64] FLAG: --port="10250"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164966    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164971    2577 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0d70d3c41c368c420"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164976    2577 flags.go:64] FLAG: --qos-reserved=""
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164981    2577 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164986    2577 flags.go:64] FLAG: --register-node="true"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164991    2577 flags.go:64] FLAG: --register-with-taints=""
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.164997    2577 flags.go:64] FLAG: --registry-burst="10"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165002    2577 flags.go:64] FLAG: --registry-qps="5"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165006    2577 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165022    2577 flags.go:64] FLAG: --reserved-memory=""
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165029    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165034    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165039    2577 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 07:28:28.171455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165044    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165048    2577 flags.go:64] FLAG: --runonce="false"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165053    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165058    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165063    2577 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165068    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165073    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165079    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165084    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165089    2577 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165094    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165098    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165105    2577 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165110    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165115    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165120    2577 flags.go:64] FLAG: --system-cgroups=""
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165124    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165133    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165137    2577 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165142    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165148    2577 flags.go:64] FLAG: --tls-min-version=""
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165153    2577 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165157    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165162    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165167    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 07:28:28.172151 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165172    2577 flags.go:64] FLAG: --v="2"
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165179    2577 flags.go:64] FLAG: --version="false"
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165185    2577 flags.go:64] FLAG: --vmodule=""
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165192    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165205    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165432    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165438    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165443    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165447    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165452    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165456    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165476    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165480    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165484    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165491    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165495    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165500    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165504    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165508    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165514    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.172764 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165519    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165523    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165528    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165532    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165537    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165541    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165545    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165549    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165554    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165558    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165562    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165566    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165570    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165575    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165581    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165588    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165593    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165600    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165606    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.173271 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165611    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165616    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165621    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165626    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165630    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165634    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165639    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165645    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165649    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165654    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165658    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165662    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165668    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165673    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165677    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165681    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165686    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165691    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165695    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.173736 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165699    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165704    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165708    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165713    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165717    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165721    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165725    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165729    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165734    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165738    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165742    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165747    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165752    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165756    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165761    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165765    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165769    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165773    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165778    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165782    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.174194 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165789    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165793    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165797    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165802    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165806    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165811    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165816    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165820    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165825    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165829    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165833    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165837    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165842    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165846    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165851    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165855    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165859    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165863    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165867    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165872    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.174678 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165876    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165881    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165885    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165891    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165897    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165903    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165908    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165912    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165916    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165920    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165925    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165929    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.165934    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.165977    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166044    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166051    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.175613 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166057    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166062    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166068    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166073    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166077    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166081    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166086    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166089    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166094    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166099    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166103    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166107    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166111    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166117    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166121    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166125    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166130    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166134    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166139    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166143    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.175997 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166147    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166152    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166157    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166161    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166165    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166169    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166176    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166182    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166189    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166193    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166197    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166202    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166206    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166211    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166218    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166222    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166226    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166231    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166235    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.176480 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166239    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166243    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166248    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166252    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166256    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166261    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166265    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166270    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166274    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166278    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166283    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166287    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166291    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166295    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166302    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166306    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166313    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166317    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166321    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.176948 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166326    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166330    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166335    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166340    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166344    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166348    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166353    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166357    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166363    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166368    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166372    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166376    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166380    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166384    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166389    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166394    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166398    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166402    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166407    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166411    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.177388 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166416    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166420    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166424    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166428    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166432    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166437    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166441    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166446    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166450    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166454    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166474    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166479    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166483    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166487    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166493    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166500    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166507    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166513    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166531    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166535    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.177874 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166541    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166546    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166550    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166555    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166559    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.166563    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.173369    2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.173383    2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.173399    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.173404    2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173505    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173511    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173514    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173518    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173521    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173523    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173526    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173529    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173532    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.178353 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173534    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173537    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173540    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173548    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173551    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173554    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173558    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173561    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173563    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173566    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173568    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173571    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173573    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173576    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173579    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173583    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173587    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173590    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173593    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173596    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.178839 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173599    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173602    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173605    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173608    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173610    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173613    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173615    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173618    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173621    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173623    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173626    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173629    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173631    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173634    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173636    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173639    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173647    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173650    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173653    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173655    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.179302 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173658    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173660    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173663    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173666    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173669    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173671    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173674    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173677    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173679    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173682    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173684    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173687    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173690    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173693    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173696    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173698    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173701    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173704    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173706    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.179822 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173709    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173712    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173714    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173717    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173720    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173722    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173725    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173727    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173730    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173732    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173742    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173745    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173748    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173750    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173753    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173756    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173758    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173761    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173763    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.180517 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173766    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173768    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173771    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173773    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173775    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173778    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173781    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173784    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173787    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173797    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173801    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173803    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173806    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173809    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173812    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173815    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173818    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173821    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.181079 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.173823    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.173851    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174061    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174072    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174075    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174078    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174086    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174089    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174091    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174094    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174097    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174099    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174102    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174104    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174107    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174109    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.181534 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174112    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174114    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174117    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174119    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174122    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174125    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174129    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174133    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174136    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174139    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174141    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174144    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174146    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174149    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174152    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174154    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174156    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174159    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174161    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.181986 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174164    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174167    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174169    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174171    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174181    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174184    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174187    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174189    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174192    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174195    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174199    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174202    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174205    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174208    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174210    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174213    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174216    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174219    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174221    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.182489 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174224    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174226    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174229    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174231    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174234    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174237    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174239    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174242    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174244    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174247    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174249    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174252    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174254    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174257    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174259    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174262    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174264    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174267    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174275    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174278    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.182940 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174280    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174283    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174285    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174288    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174291    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174293    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174295    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174298    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174300    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174303    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174309    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174312    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174314    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174316    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174319    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174321    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174324    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174326    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174329    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174331    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.183422 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174334    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174336    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174339    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174341    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174344    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174346    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174349    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174351    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174354    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174356    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174359    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174367    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174370    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174372    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.174396    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174444    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:28:28.183892 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174449    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174451    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174454    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174457    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174459    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174476    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174479    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174481    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174485    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174489    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174492    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174495    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174497    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174500    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174503    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174505    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174508    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174510    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174516    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:28:28.184308 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174519    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174521    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174524    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174527    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174529    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174532    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174535    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174537    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174540    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174548    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174551    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174556    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174560    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174563    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174566    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174568    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174571    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174573    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174576    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:28:28.184988 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174579    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174581    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174584    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174586    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174589    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174591    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174594    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174597    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174599    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174602    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174604    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174607    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174611    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174613    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174616    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174618    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174621    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174624    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174627    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174629    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:28:28.185482 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174632    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174635    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174637    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174645    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174649    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174652    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174655    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174657    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174660    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174663    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174665    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174668    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174670    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174673    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174675    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174678    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174680    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174683    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174686    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174689    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:28:28.186010 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174691    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174694    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174697    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174699    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174703    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174706    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174709    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174711    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174714    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174717    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174719    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174722    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174724    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174727    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174729    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174732    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174741    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174744    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174747    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174750    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:28:28.186602 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174753    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174755    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174758    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174760    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174762    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174765    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:28.174767    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.174818    2577 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.176814    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.177676    2577 server.go:1014] "Starting client certificate rotation"
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.177782    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.177848    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.181395    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:28:28.187135 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.183167    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:28:28.193357 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.193342    2577 log.go:25] "Validated CRI v1 runtime API"
Oct 31 07:28:28.196398 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.196385    2577 log.go:25] "Validated CRI v1 image API"
Oct 31 07:28:28.197174 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.197150    2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 07:28:28.203077 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.203059    2577 fs.go:135] Filesystem UUIDs: map[02ef2c36-5ea7-4330-a1dc-2ee4996e5206:/dev/nvme0n1p4 5e685bf5-adf7-413d-bbe3-6a1c01366601:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 07:28:28.203137 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.203076    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 07:28:28.208455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.208347    2577 manager.go:217] Machine: {Timestamp:2026-10-31 07:28:28.206785022 +0000 UTC m=+0.337298378 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095696 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f7598153718f196a663ecb5f397d6 SystemUUID:ec2f7598-1537-18f1-96a6-63ecb5f397d6 BootID:13578c0c-7132-4df0-a6ad-18c68f4a8523 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:28:4b:8c:45:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:28:4b:8c:45:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:fb:70:59:9a:43 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 07:28:28.208455 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.208454    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 07:28:28.208576 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.208564    2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 07:28:28.212055 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.212030    2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 07:28:28.212186 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.212056    2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-50.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 07:28:28.212228 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.212189    2577 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 07:28:28.212228 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.212197    2577 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 07:28:28.212284 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.212233    2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:28:28.212284 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.212251    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:28:28.212901 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.212891    2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 07:28:28.216485 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.216473    2577 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 07:28:28.216611 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.216601    2577 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 07:28:28.217683 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.217675    2577 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 07:28:28.217714 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.217700    2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 07:28:28.217714 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.217711    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 07:28:28.217791 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.217721    2577 kubelet.go:405] "Adding apiserver pod source"
Oct 31 07:28:28.217791 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.217733    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 07:28:28.218078 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.218066    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:28:28.220502 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.220453    2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 07:28:28.226040 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226027    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:28:28.226698 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226686    2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 07:28:28.226738 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226718    2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 07:28:28.226890 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226878    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 07:28:28.226923 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226900    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 07:28:28.226923 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226912    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 07:28:28.226923 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226918    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 07:28:28.226998 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226925    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 07:28:28.226998 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.226931    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 07:28:28.228239 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228217    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 07:28:28.228297 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228245    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 07:28:28.228297 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228266    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 07:28:28.228297 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228279    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 07:28:28.228428 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228316    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 07:28:28.228428 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228333    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 07:28:28.228894 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228877    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 07:28:28.228958 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.228898    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 07:28:28.233249 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.233235    2577 server.go:1263] "Started kubelet"
Oct 31 07:28:28.233371 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.233318    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 07:28:28.233411 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.233391    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 07:28:28.233747 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.233708    2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 07:28:28.233967 ip-10-0-133-50 systemd[1]: Started Kubernetes Kubelet.
Oct 31 07:28:28.235061 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.235049    2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 07:28:28.238678 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.238656    2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 07:28:28.238747 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.238677    2577 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 07:28:28.238816 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.238668    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 07:28:28.238916 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.238839    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:28.239354 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.239337    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 07:28:28.239898 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.239880    2577 factory.go:55] Registering systemd factory
Oct 31 07:28:28.240000 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.239919    2577 factory.go:223] Registration of the systemd container factory successfully
Oct 31 07:28:28.240058 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.240043    2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 07:28:28.240101 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240057    2577 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 07:28:28.240101 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240075    2577 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 07:28:28.240195 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240113    2577 factory.go:153] Registering CRI-O factory
Oct 31 07:28:28.240195 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240126    2577 factory.go:223] Registration of the crio container factory successfully
Oct 31 07:28:28.240195 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240169    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 07:28:28.240195 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240172    2577 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 07:28:28.240195 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240180    2577 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 07:28:28.240195 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240186    2577 factory.go:103] Registering Raw factory
Oct 31 07:28:28.240457 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.240201    2577 manager.go:1199] Started watching for new ooms in manager
Oct 31 07:28:28.241115 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.241099    2577 manager.go:319] Starting recovery of all containers
Oct 31 07:28:28.251942 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.251925    2577 manager.go:324] Recovery completed
Oct 31 07:28:28.256154 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.254343    2577 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Oct 31 07:28:28.259371 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.259358    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.260983 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.260960    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.261044 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.261004    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.261044 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.261021    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.261616 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.261603    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.261698 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.261602    2577 cpu_manager.go:226] "Starting" policy="none"
Oct 31 07:28:28.261698 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.261633    2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 07:28:28.261698 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.261652    2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 07:28:28.262258 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.262243    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.262324 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.262266    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.262324 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.262276    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.263918 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.263907    2577 policy_none.go:50] "Start"
Oct 31 07:28:28.263968 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.263921    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 07:28:28.263968 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.263930    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 07:28:28.265889 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.265878    2577 policy_none.go:44] "Start"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.301179    2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.303960    2577 manager.go:341] "Starting Device Plugin manager"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.304018    2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.304027    2577 server.go:87] "Starting device plugin registration server"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.304233    2577 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.304242    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.304338    2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.304418    2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.304425    2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.305042    2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.305093    2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.311050    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.312169    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.312187    2577 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.312202    2577 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.312235    2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.376615    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-50.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 07:28:28.398950 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.376598    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-50.us-west-1.compute.internal.18e38ca2fa298c22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-50.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-50.us-west-1.compute.internal,},FirstTimestamp:2026-10-31 07:28:28.233206818 +0000 UTC m=+0.363720168,LastTimestamp:2026-10-31 07:28:28.233206818 +0000 UTC m=+0.363720168,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-50.us-west-1.compute.internal,}"
Oct 31 07:28:28.405121 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.405070    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.405872 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.405855    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.405970 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.405882    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.405970 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.405894    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.405970 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.405919    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.412743 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.412725    2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"]
Oct 31 07:28:28.412806 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.412776    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.413489 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.413473    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.413577 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.413501    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.413577 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.413510    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.415608 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.415593    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.415756 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.415738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.415838 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.415772    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.416242 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.416224    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.416336 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.416231    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.416336 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.416254    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.416336 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.416268    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.416336 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.416274    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.416336 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.416292    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.418291 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.418277    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.418391 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.418299    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.418981 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.418958    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.418981 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.418983    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.419109 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.418996    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.424980 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.424952    2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-50.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 07:28:28.441209 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.441194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.441263 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.441219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.441263 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.441238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0fc2efca0f9b45f2393e3897a932b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"0fc2efca0f9b45f2393e3897a932b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.441839 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.441824    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.443783 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.443765    2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.465816 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.465787    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 07:28:28.465896 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.465787    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 31 07:28:28.465896 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.465739    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-50.us-west-1.compute.internal.18e38ca2fbd161ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-50.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-50.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-50.us-west-1.compute.internal,},FirstTimestamp:2026-10-31 07:28:28.260983279 +0000 UTC m=+0.391496631,LastTimestamp:2026-10-31 07:28:28.260983279 +0000 UTC m=+0.391496631,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-50.us-west-1.compute.internal,}"
Oct 31 07:28:28.514318 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.514296    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Oct 31 07:28:28.514396 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.514364    2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-50.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 31 07:28:28.524365 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.524351    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 07:28:28.524415 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.524386    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d2vx2"
Oct 31 07:28:28.532969 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.532907    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-50.us-west-1.compute.internal.18e38ca2fbd1d148  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-50.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-50.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-50.us-west-1.compute.internal,},FirstTimestamp:2026-10-31 07:28:28.261011784 +0000 UTC m=+0.391525134,LastTimestamp:2026-10-31 07:28:28.261011784 +0000 UTC m=+0.391525134,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-50.us-west-1.compute.internal,}"
Oct 31 07:28:28.533059 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.533006    2577 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-50.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.541631 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.541611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.541695 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.541637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.541695 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.541654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0fc2efca0f9b45f2393e3897a932b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"0fc2efca0f9b45f2393e3897a932b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.541695 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.541678    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0fc2efca0f9b45f2393e3897a932b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"0fc2efca0f9b45f2393e3897a932b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.541826 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.541704    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.541826 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.541709    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.729942 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.729892    2577 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-50.us-west-1.compute.internal\" not found" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.734007 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.733988    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:28.735103 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.735088    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:28.735181 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.735114    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:28.735181 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.735124    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:28.735181 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.735148    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.744782 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.744767    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.746926 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.746909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.790858 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:28.790729    2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:28.790939 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.790860    2577 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-50.us-west-1.compute.internal\": node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:28.909303 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:28.909277    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:29.010351 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:29.010306    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:29.110727 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:29.110694    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:29.179722 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.179703    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 07:28:29.180140 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:29.179845    2577 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://adf9bd3603f5d4feb94c1d7340fbfba2-7a76273f21d70fcd.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-133-50.us-west-1.compute.internal.18e38ca2fbd20ea2\": read tcp 10.0.133.50:55024->3.151.99.242:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-133-50.us-west-1.compute.internal.18e38ca2fbd20ea2  default   4927 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-50.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-50.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-50.us-west-1.compute.internal,},FirstTimestamp:2026-10-31 07:28:28 +0000 UTC,LastTimestamp:2026-10-31 07:28:28.405898044 +0000 UTC m=+0.536411394,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-50.us-west-1.compute.internal,}"
Oct 31 07:28:29.212436 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:29.212413    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:29.312816 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:29.312765    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:29.414897 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:29.414860    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:29.426877 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.426858    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:28:29.427667 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.427646    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:28:29.427758 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.427680    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:28:29.427758 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.427690    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:28:29.515798 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:29.515771    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:28:29.528894 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.528856    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 07:23:29 +0000 UTC" deadline="2028-08-16 01:22:09.348979412 +0000 UTC"
Oct 31 07:28:29.528894 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.528891    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15713h53m39.820091518s"
Oct 31 07:28:29.547884 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.546459    2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:29.602226 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.602174    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:29.639457 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.639435    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:29.703360 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.703344    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:28:29.705173 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.705157    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:28:29.766139 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.766113    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:28:29.839764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.839727    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:29.882307 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:29.882274    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc2efca0f9b45f2393e3897a932b4b4.slice/crio-5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01 WatchSource:0}: Error finding container 5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01: Status 404 returned error can't find the container with id 5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01
Oct 31 07:28:29.889366 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:29.889341    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:28:29.891885 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:29.891862    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09f5b3d9a6222e71966c227a0ba0115.slice/crio-28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58 WatchSource:0}: Error finding container 28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58: Status 404 returned error can't find the container with id 28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58
Oct 31 07:28:30.089672 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.089649    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:28:30.220385 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.220331    2577 apiserver.go:52] "Watching apiserver"
Oct 31 07:28:30.240076 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.240053    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 07:28:30.317633 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.317588    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"c09f5b3d9a6222e71966c227a0ba0115","Type":"ContainerStarted","Data":"28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58"}
Oct 31 07:28:30.318674 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.318654    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"0fc2efca0f9b45f2393e3897a932b4b4","Type":"ContainerStarted","Data":"5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01"}
Oct 31 07:28:30.330175 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.330154    2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 07:28:30.330457 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.330441    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-zgfmt","kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh","openshift-image-registry/node-ca-w5t6j","openshift-cluster-node-tuning-operator/tuned-7mhkh","openshift-multus/multus-additional-cni-plugins-c6r54","openshift-multus/network-metrics-daemon-w7gt6","openshift-network-diagnostics/network-check-target-7cl42","kube-system/global-pull-secret-syncer-6qkpn","openshift-dns/node-resolver-4k94x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal","openshift-multus/multus-b76wl","openshift-network-operator/iptables-alerter-jk2nc","openshift-ovn-kubernetes/ovnkube-node-94t84"]
Oct 31 07:28:30.334181 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.334164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:30.334259 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.334226    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:30.336352 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.336339    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.338633 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.338617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.340996 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.340979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.343302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.343289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.345815 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.345798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.348204 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.348187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.348281 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.348251    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:30.350592 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.350576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.351121 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ca8ac6-dfed-461e-90ed-17556a2e5272-host-slash\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.351202 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-socket-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.351202 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-registration-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.351202 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb6tz\" (UniqueName: \"kubernetes.io/projected/98e9b442-22a5-4329-9a1f-872e81ff6505-kube-api-access-qb6tz\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.351378 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ca8ac6-dfed-461e-90ed-17556a2e5272-iptables-alerter-script\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.351378 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-etc-selinux\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.351378 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e9b442-22a5-4329-9a1f-872e81ff6505-host\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.351563 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.351563 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-device-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.351563 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-sys-fs\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.351563 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99wxd\" (UniqueName: \"kubernetes.io/projected/46ca8ac6-dfed-461e-90ed-17556a2e5272-kube-api-access-99wxd\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.351563 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e9b442-22a5-4329-9a1f-872e81ff6505-serviceca\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.351775 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351564    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:30.351775 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nprzm\" (UniqueName: \"kubernetes.io/projected/09202313-5fd7-46ab-b667-18188c9b28e7-kube-api-access-nprzm\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:30.351775 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.351657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znp8b\" (UniqueName: \"kubernetes.io/projected/9b15479c-ad1d-4bae-af69-9409ce12337e-kube-api-access-znp8b\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.353375 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.353359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.356377 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.356358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.358932 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.358918    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:30.361690 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.359245    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:30.361924 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.361907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.396408 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.396388    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:28:30.396494 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.396414    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 07:28:30.396494 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.396419    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 07:28:30.397345 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.397325    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 07:28:30.397345 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.397332    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 07:28:30.397464 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.397394    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 07:28:30.397464 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.397407    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 07:28:30.397594 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.397538    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-46dh7\""
Oct 31 07:28:30.397594 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.397565    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-798mv\""
Oct 31 07:28:30.397594 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.397576    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kwsqb\""
Oct 31 07:28:30.407242 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.407225    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 07:28:30.407349 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.407303    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 07:28:30.407349 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.407312    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 07:28:30.407947 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.407925    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 07:28:30.408282 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408267    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 07:28:30.408384 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408370    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 07:28:30.408538 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408523    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 07:28:30.408604 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408576    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:28:30.408604 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408582    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 07:28:30.408604 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408587    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jtbjn\""
Oct 31 07:28:30.408725 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408630    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 07:28:30.408725 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408642    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pvmcp\""
Oct 31 07:28:30.408725 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408686    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w8pvl\""
Oct 31 07:28:30.408826 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.408781    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gttml\""
Oct 31 07:28:30.409109 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.409094    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 07:28:30.409146 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.409110    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 07:28:30.409214 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.409201    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 07:28:30.409248 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.409233    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 07:28:30.409764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.409750    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 07:28:30.411848 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.411834    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 07:28:30.412374 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.412362    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 07:28:30.414220 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.414206    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 07:28:30.439369 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.439347    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7fk4f\""
Oct 31 07:28:30.439369 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.439349    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 07:28:30.440084 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.440070    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 07:28:30.441560 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.441546    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 07:28:30.443315 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.443299    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 07:28:30.443366 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.443306    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-ldrdc\""
Oct 31 07:28:30.443366 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.443329    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-45qbl"
Oct 31 07:28:30.452079 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-run\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.452136 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.452136 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znp8b\" (UniqueName: \"kubernetes.io/projected/9b15479c-ad1d-4bae-af69-9409ce12337e-kube-api-access-znp8b\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.452136 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ca8ac6-dfed-461e-90ed-17556a2e5272-host-slash\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.452239 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-socket-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.452332 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-kubernetes\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.452332 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452316    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ca8ac6-dfed-461e-90ed-17556a2e5272-host-slash\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.452401 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452348    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-socket-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.452401 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nghgg\" (UniqueName: \"kubernetes.io/projected/7b3156c0-bb65-443c-bfde-3ed0151d47ce-kube-api-access-nghgg\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.452401 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qxq2\" (UniqueName: \"kubernetes.io/projected/30297c93-42e8-45c4-91b4-bc87be152069-kube-api-access-7qxq2\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.452502 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adedcfe7-c4f4-417e-ba6d-af46396d04f8-tmp-dir\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.452502 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-var-lib-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.452502 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-ovn\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.452502 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e9b442-22a5-4329-9a1f-872e81ff6505-host\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb63cc-bc6e-424c-a528-5232fdfc561c-agent-certs\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452592    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e9b442-22a5-4329-9a1f-872e81ff6505-host\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-system-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452620    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-daemon-config\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-systemd-units\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.452656 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:30.452916 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e9b442-22a5-4329-9a1f-872e81ff6505-serviceca\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.452916 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-tmp\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.452916 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-etc-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.452916 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.452916 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452808    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-socket-dir-parent\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.452916 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-multus-certs\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.452916 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-etc-kubernetes\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-kubelet\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.452976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-bin\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-netd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-config\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-env-overrides\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453094    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e9b442-22a5-4329-9a1f-872e81ff6505-serviceca\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/adedcfe7-c4f4-417e-ba6d-af46396d04f8-hosts-file\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.453149 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-conf-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453155    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-lib-modules\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-log-socket\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453234    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cni-binary-copy\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-device-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453326    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-system-cni-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.453399 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453381    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-device-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nprzm\" (UniqueName: \"kubernetes.io/projected/09202313-5fd7-46ab-b667-18188c9b28e7-kube-api-access-nprzm\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-bin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-netns\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-script-lib\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb6tz\" (UniqueName: \"kubernetes.io/projected/98e9b442-22a5-4329-9a1f-872e81ff6505-kube-api-access-qb6tz\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cnibin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-multus\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjnhw\" (UniqueName: \"kubernetes.io/projected/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-kube-api-access-bjnhw\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-tuned\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-kubelet-config\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.453751 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-slash\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ca8ac6-dfed-461e-90ed-17556a2e5272-iptables-alerter-script\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-etc-selinux\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.453978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n299\" (UniqueName: \"kubernetes.io/projected/aa0b9899-ff27-48dc-85a3-e88c23f8e897-kube-api-access-4n299\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454010    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-etc-selinux\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-sys-fs\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.454106 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysconfig\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-systemd\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454116    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-sys-fs\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99wxd\" (UniqueName: \"kubernetes.io/projected/46ca8ac6-dfed-461e-90ed-17556a2e5272-kube-api-access-99wxd\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-cnibin\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-registration-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-kubelet\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454323    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ca8ac6-dfed-461e-90ed-17556a2e5272-iptables-alerter-script\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.454414 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-hostroot\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.454763 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-sys\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.454850 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.454832    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:30.454908 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-host\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.454966 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-node-log\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.454966 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.454951    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:30.954929587 +0000 UTC m=+3.085442955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:30.454966 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454957    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-registration-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.455077 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.454989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-binary-copy\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.455077 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-systemd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.455077 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovn-node-metrics-cert\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.455077 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-conf\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.455077 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-dbus\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.455227 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-var-lib-kubelet\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.455227 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb63cc-bc6e-424c-a528-5232fdfc561c-konnectivity-ca\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.455227 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpf64\" (UniqueName: \"kubernetes.io/projected/adedcfe7-c4f4-417e-ba6d-af46396d04f8-kube-api-access-fpf64\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.455227 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-k8s-cni-cncf-io\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.455227 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-netns\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.455381 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.455226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-modprobe-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.512343 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.512282    2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 07:28:30.516049 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.516024    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb6tz\" (UniqueName: \"kubernetes.io/projected/98e9b442-22a5-4329-9a1f-872e81ff6505-kube-api-access-qb6tz\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.516129 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.516032    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znp8b\" (UniqueName: \"kubernetes.io/projected/9b15479c-ad1d-4bae-af69-9409ce12337e-kube-api-access-znp8b\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.516129 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.516081    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99wxd\" (UniqueName: \"kubernetes.io/projected/46ca8ac6-dfed-461e-90ed-17556a2e5272-kube-api-access-99wxd\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.516201 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.516144    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nprzm\" (UniqueName: \"kubernetes.io/projected/09202313-5fd7-46ab-b667-18188c9b28e7-kube-api-access-nprzm\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:30.556113 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-tmp\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.556206 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-etc-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556206 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.556206 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-socket-dir-parent\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556206 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556195    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-etc-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-multus-certs\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-etc-kubernetes\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556247    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-multus-certs\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-kubelet\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-bin\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556286    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-etc-kubernetes\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556315    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-bin\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-netd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-netd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556343    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-socket-dir-parent\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-config\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556370    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-kubelet\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-env-overrides\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/adedcfe7-c4f4-417e-ba6d-af46396d04f8-hosts-file\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-conf-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-lib-modules\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556491    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556499    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/adedcfe7-c4f4-417e-ba6d-af46396d04f8-hosts-file\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-log-socket\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556536    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-conf-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cni-binary-copy\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556614    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-lib-modules\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-system-cni-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556632    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-log-socket\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556671    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-system-cni-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-bin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-netns\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556741    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-bin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556739    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.556960 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-script-lib\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.556780    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-netns\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.557092    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cnibin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557211    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557258    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cnibin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557264    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-multus\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-multus\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.557298    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret podName:65d23093-9f6e-4857-90be-3504d74e6dde nodeName:}" failed. No retries permitted until 2026-10-31 07:28:31.057278878 +0000 UTC m=+3.187792227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret") pod "global-pull-secret-syncer-6qkpn" (UID: "65d23093-9f6e-4857-90be-3504d74e6dde") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557328    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-env-overrides\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjnhw\" (UniqueName: \"kubernetes.io/projected/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-kube-api-access-bjnhw\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-tuned\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557371    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cni-binary-copy\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-kubelet-config\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557419    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-slash\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557469    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-config\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557472    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-script-lib\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.557658 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557502    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-slash\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557569    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-kubelet-config\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n299\" (UniqueName: \"kubernetes.io/projected/aa0b9899-ff27-48dc-85a3-e88c23f8e897-kube-api-access-4n299\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557592    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysconfig\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557642    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-systemd\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557709    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysconfig\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557735    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-systemd\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-cnibin\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557784    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-cnibin\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-kubelet\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557807    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-kubelet\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-hostroot\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-sys\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557870    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-hostroot\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.558520 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-host\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557902    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-node-log\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557925    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-sys\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-binary-copy\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-systemd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557974    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-node-log\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557975    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-host\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.557988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovn-node-metrics-cert\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-conf\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558031    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-systemd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-dbus\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558104    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558116    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-dbus\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-var-lib-kubelet\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb63cc-bc6e-424c-a528-5232fdfc561c-konnectivity-ca\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558199    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-conf\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpf64\" (UniqueName: \"kubernetes.io/projected/adedcfe7-c4f4-417e-ba6d-af46396d04f8-kube-api-access-fpf64\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.559302 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-k8s-cni-cncf-io\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-netns\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-modprobe-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-run\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558324    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-var-lib-kubelet\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-kubernetes\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nghgg\" (UniqueName: \"kubernetes.io/projected/7b3156c0-bb65-443c-bfde-3ed0151d47ce-kube-api-access-nghgg\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qxq2\" (UniqueName: \"kubernetes.io/projected/30297c93-42e8-45c4-91b4-bc87be152069-kube-api-access-7qxq2\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558444    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-modprobe-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558449    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-tmp\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558478    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-k8s-cni-cncf-io\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558516    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-run\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adedcfe7-c4f4-417e-ba6d-af46396d04f8-tmp-dir\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558566    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-var-lib-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558569    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-kubernetes\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558587    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-netns\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.560143 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-ovn\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558634    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-var-lib-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558665    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-ovn\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558698    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb63cc-bc6e-424c-a528-5232fdfc561c-agent-certs\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-system-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558768    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-daemon-config\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.558786    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adedcfe7-c4f4-417e-ba6d-af46396d04f8-tmp-dir\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.559273    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-system-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.559387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-systemd-units\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.559423    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb63cc-bc6e-424c-a528-5232fdfc561c-konnectivity-ca\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.559425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.559509    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-binary-copy\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.559720    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-tuned\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.560984 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.559768    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-systemd-units\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.561803 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.560921    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovn-node-metrics-cert\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.563115 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.563095    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-daemon-config\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.563187 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.563160    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb63cc-bc6e-424c-a528-5232fdfc561c-agent-certs\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.617785 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.617762    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:30.617785 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.617790    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:30.617959 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.617807    2577 projected.go:196] Error preparing data for projected volume kube-api-access-5bfvr for pod openshift-network-diagnostics/network-check-target-7cl42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:30.617959 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.617869    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr podName:d4baf7ce-ca52-4d1b-a721-39ba6be4647f nodeName:}" failed. No retries permitted until 2026-10-31 07:28:31.117849374 +0000 UTC m=+3.248362736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5bfvr" (UniqueName: "kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr") pod "network-check-target-7cl42" (UID: "d4baf7ce-ca52-4d1b-a721-39ba6be4647f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:30.618838 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.618805    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjnhw\" (UniqueName: \"kubernetes.io/projected/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-kube-api-access-bjnhw\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.618948 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.618908    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpf64\" (UniqueName: \"kubernetes.io/projected/adedcfe7-c4f4-417e-ba6d-af46396d04f8-kube-api-access-fpf64\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.619387 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.619366    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n299\" (UniqueName: \"kubernetes.io/projected/aa0b9899-ff27-48dc-85a3-e88c23f8e897-kube-api-access-4n299\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.619472 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.619454    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nghgg\" (UniqueName: \"kubernetes.io/projected/7b3156c0-bb65-443c-bfde-3ed0151d47ce-kube-api-access-nghgg\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.619778 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.619759    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qxq2\" (UniqueName: \"kubernetes.io/projected/30297c93-42e8-45c4-91b4-bc87be152069-kube-api-access-7qxq2\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.649673 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.649647    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:28:30.659590 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.659565    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ca8ac6_dfed_461e_90ed_17556a2e5272.slice/crio-583c98cf4ca216c2bcb77d9e45d3d6cbeacbf83325e0af838793af9a4fe02375 WatchSource:0}: Error finding container 583c98cf4ca216c2bcb77d9e45d3d6cbeacbf83325e0af838793af9a4fe02375: Status 404 returned error can't find the container with id 583c98cf4ca216c2bcb77d9e45d3d6cbeacbf83325e0af838793af9a4fe02375
Oct 31 07:28:30.660285 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.660266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:28:30.666402 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.666384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:28:30.668570 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.668534    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b15479c_ad1d_4bae_af69_9409ce12337e.slice/crio-395e5c0f873407f0f6afad75c577b5435b5d9c19f9be4048b567325b58fb447a WatchSource:0}: Error finding container 395e5c0f873407f0f6afad75c577b5435b5d9c19f9be4048b567325b58fb447a: Status 404 returned error can't find the container with id 395e5c0f873407f0f6afad75c577b5435b5d9c19f9be4048b567325b58fb447a
Oct 31 07:28:30.669534 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.669516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:28:30.676152 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.676129    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e9b442_22a5_4329_9a1f_872e81ff6505.slice/crio-8dbd7dfc6c19def94c43ca469fec4670c8c38ffa3454b40a67cc1cc84f07271f WatchSource:0}: Error finding container 8dbd7dfc6c19def94c43ca469fec4670c8c38ffa3454b40a67cc1cc84f07271f: Status 404 returned error can't find the container with id 8dbd7dfc6c19def94c43ca469fec4670c8c38ffa3454b40a67cc1cc84f07271f
Oct 31 07:28:30.679114 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.679089    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3156c0_bb65_443c_bfde_3ed0151d47ce.slice/crio-d76f3e74fc50c6e0d01b8c8655fae4eee22ff8df92da0171d9f6a4ce214e4cf9 WatchSource:0}: Error finding container d76f3e74fc50c6e0d01b8c8655fae4eee22ff8df92da0171d9f6a4ce214e4cf9: Status 404 returned error can't find the container with id d76f3e74fc50c6e0d01b8c8655fae4eee22ff8df92da0171d9f6a4ce214e4cf9
Oct 31 07:28:30.686776 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.686759    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:30.692777 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.692762    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:28:30.699627 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.699608    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:28:30.701584 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.701559    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b76wl"
Oct 31 07:28:30.701704 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.701679    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30297c93_42e8_45c4_91b4_bc87be152069.slice/crio-c19671c9797d3022d0ec1596954989a36cd362ea11f6d2231ad3e94b7608aefa WatchSource:0}: Error finding container c19671c9797d3022d0ec1596954989a36cd362ea11f6d2231ad3e94b7608aefa: Status 404 returned error can't find the container with id c19671c9797d3022d0ec1596954989a36cd362ea11f6d2231ad3e94b7608aefa
Oct 31 07:28:30.709743 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.709723    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadedcfe7_c4f4_417e_ba6d_af46396d04f8.slice/crio-23a52a8681039cf98cc7053c4639885ecbdfbb6a0cfd9c34446efc7db25d74db WatchSource:0}: Error finding container 23a52a8681039cf98cc7053c4639885ecbdfbb6a0cfd9c34446efc7db25d74db: Status 404 returned error can't find the container with id 23a52a8681039cf98cc7053c4639885ecbdfbb6a0cfd9c34446efc7db25d74db
Oct 31 07:28:30.712379 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.712356    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ce3135_50d9_4dd2_b8e5_44af7397ec68.slice/crio-0fa0de5fc59ed79c13caa15a341e48b93450e339420b8151da16d13f476bee10 WatchSource:0}: Error finding container 0fa0de5fc59ed79c13caa15a341e48b93450e339420b8151da16d13f476bee10: Status 404 returned error can't find the container with id 0fa0de5fc59ed79c13caa15a341e48b93450e339420b8151da16d13f476bee10
Oct 31 07:28:30.712453 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.712406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:30.722464 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:28:30.722439    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0b9899_ff27_48dc_85a3_e88c23f8e897.slice/crio-f3b4918a8db6722bff832095e27346be39aa09fc3f6e3ad57d40d5a9c198c614 WatchSource:0}: Error finding container f3b4918a8db6722bff832095e27346be39aa09fc3f6e3ad57d40d5a9c198c614: Status 404 returned error can't find the container with id f3b4918a8db6722bff832095e27346be39aa09fc3f6e3ad57d40d5a9c198c614
Oct 31 07:28:30.963304 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:30.963197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:30.963469 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.963347    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:30.963469 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:30.963393    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:31.96337936 +0000 UTC m=+4.093892712 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:31.063854 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.063819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:31.064004 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.063971    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:31.064061 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.064048    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret podName:65d23093-9f6e-4857-90be-3504d74e6dde nodeName:}" failed. No retries permitted until 2026-10-31 07:28:32.06402487 +0000 UTC m=+4.194538223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret") pod "global-pull-secret-syncer-6qkpn" (UID: "65d23093-9f6e-4857-90be-3504d74e6dde") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:31.164827 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.164790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:31.164992 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.164959    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:31.164992 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.164977    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:31.164992 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.164990    2577 projected.go:196] Error preparing data for projected volume kube-api-access-5bfvr for pod openshift-network-diagnostics/network-check-target-7cl42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:31.165129 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.165043    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr podName:d4baf7ce-ca52-4d1b-a721-39ba6be4647f nodeName:}" failed. No retries permitted until 2026-10-31 07:28:32.165026712 +0000 UTC m=+4.295540066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5bfvr" (UniqueName: "kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr") pod "network-check-target-7cl42" (UID: "d4baf7ce-ca52-4d1b-a721-39ba6be4647f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:31.321662 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.321568    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jk2nc" event={"ID":"46ca8ac6-dfed-461e-90ed-17556a2e5272","Type":"ContainerStarted","Data":"583c98cf4ca216c2bcb77d9e45d3d6cbeacbf83325e0af838793af9a4fe02375"}
Oct 31 07:28:31.323663 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.323537    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"f3b4918a8db6722bff832095e27346be39aa09fc3f6e3ad57d40d5a9c198c614"}
Oct 31 07:28:31.324944 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.324918    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b76wl" event={"ID":"b6ce3135-50d9-4dd2-b8e5-44af7397ec68","Type":"ContainerStarted","Data":"0fa0de5fc59ed79c13caa15a341e48b93450e339420b8151da16d13f476bee10"}
Oct 31 07:28:31.326331 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.326273    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4k94x" event={"ID":"adedcfe7-c4f4-417e-ba6d-af46396d04f8","Type":"ContainerStarted","Data":"23a52a8681039cf98cc7053c4639885ecbdfbb6a0cfd9c34446efc7db25d74db"}
Oct 31 07:28:31.327704 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.327680    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerStarted","Data":"c19671c9797d3022d0ec1596954989a36cd362ea11f6d2231ad3e94b7608aefa"}
Oct 31 07:28:31.329063 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.329029    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7mhkh" event={"ID":"7b3156c0-bb65-443c-bfde-3ed0151d47ce","Type":"ContainerStarted","Data":"d76f3e74fc50c6e0d01b8c8655fae4eee22ff8df92da0171d9f6a4ce214e4cf9"}
Oct 31 07:28:31.333018 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.332983    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5t6j" event={"ID":"98e9b442-22a5-4329-9a1f-872e81ff6505","Type":"ContainerStarted","Data":"8dbd7dfc6c19def94c43ca469fec4670c8c38ffa3454b40a67cc1cc84f07271f"}
Oct 31 07:28:31.334343 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.334313    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgfmt" event={"ID":"dbdb63cc-bc6e-424c-a528-5232fdfc561c","Type":"ContainerStarted","Data":"cf1faebd06b43da90ccb06bfd0490bf51eca7690822fea73336a7e1173c32523"}
Oct 31 07:28:31.335504 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.335476    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh" event={"ID":"9b15479c-ad1d-4bae-af69-9409ce12337e","Type":"ContainerStarted","Data":"395e5c0f873407f0f6afad75c577b5435b5d9c19f9be4048b567325b58fb447a"}
Oct 31 07:28:31.446764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.446699    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:23:30 +0000 UTC" deadline="2028-05-02 20:17:07.944206252 +0000 UTC"
Oct 31 07:28:31.446764 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.446753    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13188h48m36.497456979s"
Oct 31 07:28:31.974882 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:31.974838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:31.975157 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.975060    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:31.975157 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:31.975128    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:33.975109352 +0000 UTC m=+6.105622705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:32.075476 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:32.075399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:32.075895 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.075781    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:32.075895 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.075869    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret podName:65d23093-9f6e-4857-90be-3504d74e6dde nodeName:}" failed. No retries permitted until 2026-10-31 07:28:34.075848876 +0000 UTC m=+6.206362233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret") pod "global-pull-secret-syncer-6qkpn" (UID: "65d23093-9f6e-4857-90be-3504d74e6dde") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:32.177348 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:32.177316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:32.177575 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.177515    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:32.177575 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.177551    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:32.177575 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.177565    2577 projected.go:196] Error preparing data for projected volume kube-api-access-5bfvr for pod openshift-network-diagnostics/network-check-target-7cl42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:32.177802 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.177623    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr podName:d4baf7ce-ca52-4d1b-a721-39ba6be4647f nodeName:}" failed. No retries permitted until 2026-10-31 07:28:34.177604221 +0000 UTC m=+6.308117574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5bfvr" (UniqueName: "kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr") pod "network-check-target-7cl42" (UID: "d4baf7ce-ca52-4d1b-a721-39ba6be4647f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:32.316045 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:32.315970    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:32.316213 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.316123    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:32.316275 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:32.316221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:32.316327 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.316300    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:32.316380 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:32.316364    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:32.316474 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:32.316450    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:32.449222 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:32.449146    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:23:30 +0000 UTC" deadline="2028-08-15 03:07:34.435439038 +0000 UTC"
Oct 31 07:28:32.449222 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:32.449214    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15691h39m1.986230085s"
Oct 31 07:28:33.368035 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:33.367771    2577 generic.go:356] "Generic (PLEG): container finished" podID="c09f5b3d9a6222e71966c227a0ba0115" containerID="d081c3b74415386d003e3a54049d486358a1fa6b169497ee7114ff3afe4d12f5" exitCode=0
Oct 31 07:28:33.368185 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:33.368060    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"c09f5b3d9a6222e71966c227a0ba0115","Type":"ContainerDied","Data":"d081c3b74415386d003e3a54049d486358a1fa6b169497ee7114ff3afe4d12f5"}
Oct 31 07:28:33.391358 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:33.391326    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"0fc2efca0f9b45f2393e3897a932b4b4","Type":"ContainerStarted","Data":"d8b568abb584898e2462161b12a525ae7b2e274723ba0577835307c44d12b899"}
Oct 31 07:28:33.997245 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:33.997149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:33.998065 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:33.997332    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:33.998065 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:33.997398    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:37.997378332 +0000 UTC m=+10.127891689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:34.098335 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:34.098280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:34.098582 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.098561    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:34.098676 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.098645    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret podName:65d23093-9f6e-4857-90be-3504d74e6dde nodeName:}" failed. No retries permitted until 2026-10-31 07:28:38.098625389 +0000 UTC m=+10.229138742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret") pod "global-pull-secret-syncer-6qkpn" (UID: "65d23093-9f6e-4857-90be-3504d74e6dde") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:34.199088 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:34.199049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:34.199301 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.199282    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:34.199370 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.199316    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:34.199370 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.199332    2577 projected.go:196] Error preparing data for projected volume kube-api-access-5bfvr for pod openshift-network-diagnostics/network-check-target-7cl42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:34.199498 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.199484    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr podName:d4baf7ce-ca52-4d1b-a721-39ba6be4647f nodeName:}" failed. No retries permitted until 2026-10-31 07:28:38.199383465 +0000 UTC m=+10.329896821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5bfvr" (UniqueName: "kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr") pod "network-check-target-7cl42" (UID: "d4baf7ce-ca52-4d1b-a721-39ba6be4647f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:34.313355 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:34.313327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:34.313514 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.313474    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:34.313614 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:34.313560    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:34.313668 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.313632    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:34.313788 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:34.313768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:34.313947 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:34.313891    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:34.414364 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:34.414326    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"c09f5b3d9a6222e71966c227a0ba0115","Type":"ContainerStarted","Data":"ba5be1142f776978161c22bc4ed843c949d29b8bbe455ec5692568057efe856b"}
Oct 31 07:28:34.548565 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:34.535819    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal" podStartSLOduration=4.535799004 podStartE2EDuration="4.535799004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:33.611752722 +0000 UTC m=+5.742266086" watchObservedRunningTime="2026-10-31 07:28:34.535799004 +0000 UTC m=+6.666312372"
Oct 31 07:28:35.429577 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:35.429368    2577 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="9a0d8a1a1ff9cb28cbec29175a713ce64c75f512d7c49471afb6315596bd1a78" exitCode=0
Oct 31 07:28:35.430324 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:35.430060    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerDied","Data":"9a0d8a1a1ff9cb28cbec29175a713ce64c75f512d7c49471afb6315596bd1a78"}
Oct 31 07:28:35.662229 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:35.662113    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal" podStartSLOduration=5.662092131 podStartE2EDuration="5.662092131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:34.535748701 +0000 UTC m=+6.666262064" watchObservedRunningTime="2026-10-31 07:28:35.662092131 +0000 UTC m=+7.792605493"
Oct 31 07:28:36.318415 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:36.318326    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:36.318615 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:36.318472    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:36.318615 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:36.318584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:36.318738 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:36.318665    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:36.318738 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:36.318720    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:36.318846 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:36.318768    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:36.438995 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:36.438954    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgfmt" event={"ID":"dbdb63cc-bc6e-424c-a528-5232fdfc561c","Type":"ContainerStarted","Data":"e591d65769cd895aba570d55a1858b78b2a152db7a031aafd5f576a9dd4569fd"}
Oct 31 07:28:37.449162 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:37.449130    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5t6j" event={"ID":"98e9b442-22a5-4329-9a1f-872e81ff6505","Type":"ContainerStarted","Data":"eb2946db26e2ff3422f73ef00a3b49e5c8d3aa07d77bfefb6c2571412ba8038e"}
Oct 31 07:28:37.568373 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:37.568275    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zgfmt" podStartSLOduration=3.847382454 podStartE2EDuration="8.568254711s" totalImagesPullingTime="4.720872257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:36.56038279 +0000 UTC m=+8.690896157" watchObservedRunningTime="2026-10-31 07:28:37.568254711 +0000 UTC m=+9.698768077"
Oct 31 07:28:38.032680 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:38.032602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:38.032841 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.032795    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:38.032905 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.032857    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:28:46.032838544 +0000 UTC m=+18.163351885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:38.133626 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:38.133343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:38.133626 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.133534    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:38.133626 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.133592    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret podName:65d23093-9f6e-4857-90be-3504d74e6dde nodeName:}" failed. No retries permitted until 2026-10-31 07:28:46.133575374 +0000 UTC m=+18.264088724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret") pod "global-pull-secret-syncer-6qkpn" (UID: "65d23093-9f6e-4857-90be-3504d74e6dde") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:38.234071 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:38.233761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:38.234071 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.233957    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:38.234071 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.233977    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:38.234071 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.233986    2577 projected.go:196] Error preparing data for projected volume kube-api-access-5bfvr for pod openshift-network-diagnostics/network-check-target-7cl42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:38.234071 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.234027    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr podName:d4baf7ce-ca52-4d1b-a721-39ba6be4647f nodeName:}" failed. No retries permitted until 2026-10-31 07:28:46.234014033 +0000 UTC m=+18.364527382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5bfvr" (UniqueName: "kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr") pod "network-check-target-7cl42" (UID: "d4baf7ce-ca52-4d1b-a721-39ba6be4647f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:38.324935 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:38.324866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:38.325280 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:38.325144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:38.325600 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.325521    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:38.325600 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:38.325565    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:38.325862 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.325840    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:38.326038 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:38.326006    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:38.460378 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:38.460345    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh" event={"ID":"9b15479c-ad1d-4bae-af69-9409ce12337e","Type":"ContainerStarted","Data":"d2e71dc861f9e8649a0905f1ec96a4e1e28a29922863f56ae29a9a1243d257e3"}
Oct 31 07:28:39.099696 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.099176    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:39.100221 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.100047    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:39.211411 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.211326    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w5t6j" podStartSLOduration=4.44506007 podStartE2EDuration="10.211309274s" totalImagesPullingTime="5.766249204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:37.568683154 +0000 UTC m=+9.699196516" watchObservedRunningTime="2026-10-31 07:28:39.211309274 +0000 UTC m=+11.341822633"
Oct 31 07:28:39.301336 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.301284    2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 07:28:39.320478 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.319850    2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T07:28:39.301301369Z","UUID":"c7add1b4-12af-43ce-ae15-54118a17cf61","Handler":null,"Name":"","Endpoint":""}
Oct 31 07:28:39.321268 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.321239    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 07:28:39.321268 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.321266    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 07:28:39.467144 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:39.467114    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh" event={"ID":"9b15479c-ad1d-4bae-af69-9409ce12337e","Type":"ContainerStarted","Data":"c2d26743e54e8045dde12f06c977beee9dbdb3bfba0534644e4a53141738f9ef"}
Oct 31 07:28:40.314635 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.314601    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:40.314635 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.314619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:40.314848 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:40.314747    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:40.314958 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:40.314934    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:40.315054 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.314965    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:40.315054 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:40.315038    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:40.472710 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.472228    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jk2nc" event={"ID":"46ca8ac6-dfed-461e-90ed-17556a2e5272","Type":"ContainerStarted","Data":"dc2ff009e9e2c06e31fa3eb7bfb9eaedd0f9befb2109193a3c1d379118623b24"}
Oct 31 07:28:40.475304 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.475277    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b76wl" event={"ID":"b6ce3135-50d9-4dd2-b8e5-44af7397ec68","Type":"ContainerStarted","Data":"5a9dbc66dad8cbfad99f491054725da73ff8752108e7ae0b957e90663843aa44"}
Oct 31 07:28:40.477578 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.477547    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4k94x" event={"ID":"adedcfe7-c4f4-417e-ba6d-af46396d04f8","Type":"ContainerStarted","Data":"a1fd7f32fc7686c5d72ee46c7374a2fb0d41faa2578bb23bf9d25f8c2362cd40"}
Oct 31 07:28:40.480168 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.480142    2577 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="5c7ff9a477950e1cbe80f87f8e48320b25d9f0d70d57884c27d5e34364fc3bb1" exitCode=0
Oct 31 07:28:40.480257 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.480179    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerDied","Data":"5c7ff9a477950e1cbe80f87f8e48320b25d9f0d70d57884c27d5e34364fc3bb1"}
Oct 31 07:28:40.584094 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.583449    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jk2nc" podStartSLOduration=3.221153928 podStartE2EDuration="11.583429076s" totalImagesPullingTime="8.362275148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:40.582801468 +0000 UTC m=+12.713314824" watchObservedRunningTime="2026-10-31 07:28:40.583429076 +0000 UTC m=+12.713942434"
Oct 31 07:28:40.808137 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.808086    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-4k94x" podStartSLOduration=3.035477787 podStartE2EDuration="11.808066579s" totalImagesPullingTime="8.772588792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:40.807309244 +0000 UTC m=+12.937822602" watchObservedRunningTime="2026-10-31 07:28:40.808066579 +0000 UTC m=+12.938579938"
Oct 31 07:28:40.921099 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:40.921000    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-b76wl" podStartSLOduration=3.236991817 podStartE2EDuration="11.920987332s" totalImagesPullingTime="8.683995515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:40.920871656 +0000 UTC m=+13.051385014" watchObservedRunningTime="2026-10-31 07:28:40.920987332 +0000 UTC m=+13.051500689"
Oct 31 07:28:41.485115 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:41.485079    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh" event={"ID":"9b15479c-ad1d-4bae-af69-9409ce12337e","Type":"ContainerStarted","Data":"bab47de7334e30f9f15f85000367c2932153d3e3557ef45b9b063a2b9fc108c6"}
Oct 31 07:28:42.313513 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.313480    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:42.313704 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.313553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:42.313704 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.313553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:42.313704 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:42.313689    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:42.313858 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:42.313742    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:42.313858 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:42.313804    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:42.489441 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.489408    2577 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="7ea6fc44a5f9c4c098690cc4acf3ac9e49bf61ecdb3e13c4e26546d304bca88d" exitCode=0
Oct 31 07:28:42.490037 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.489505    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerDied","Data":"7ea6fc44a5f9c4c098690cc4acf3ac9e49bf61ecdb3e13c4e26546d304bca88d"}
Oct 31 07:28:42.491500 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.491142    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7mhkh" event={"ID":"7b3156c0-bb65-443c-bfde-3ed0151d47ce","Type":"ContainerStarted","Data":"f82067dd5f842448704a977324ee58ad2c4140b842c62e6fc72afde253096cf2"}
Oct 31 07:28:42.606120 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.606022    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh" podStartSLOduration=3.603680528 podStartE2EDuration="13.606003189s" totalImagesPullingTime="10.002322661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:41.599531664 +0000 UTC m=+13.730045025" watchObservedRunningTime="2026-10-31 07:28:42.606003189 +0000 UTC m=+14.736516549"
Oct 31 07:28:42.717911 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:42.717850    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7mhkh" podStartSLOduration=2.817716495 podStartE2EDuration="13.717830531s" totalImagesPullingTime="10.900114036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:42.717439138 +0000 UTC m=+14.847952497" watchObservedRunningTime="2026-10-31 07:28:42.717830531 +0000 UTC m=+14.848343892"
Oct 31 07:28:44.312937 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:44.312909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:44.313549 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:44.312959    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:44.313549 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:44.312973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:44.313549 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:44.313052    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:44.313549 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:44.313144    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:44.313549 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:44.313195    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:44.497372 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:44.497335    2577 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="1f4d6dc3b57cd395c87d329e7752110a512e45987f769597deb04a4aa5aab464" exitCode=0
Oct 31 07:28:44.497539 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:44.497401    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerDied","Data":"1f4d6dc3b57cd395c87d329e7752110a512e45987f769597deb04a4aa5aab464"}
Oct 31 07:28:46.097330 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.097108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:46.097757 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.097251    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:46.097757 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.097483    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:02.09744986 +0000 UTC m=+34.227963217 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 07:28:46.198725 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.198689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:46.198900 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.198846    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:46.198967 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.198919    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret podName:65d23093-9f6e-4857-90be-3504d74e6dde nodeName:}" failed. No retries permitted until 2026-10-31 07:29:02.198901638 +0000 UTC m=+34.329414990 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret") pod "global-pull-secret-syncer-6qkpn" (UID: "65d23093-9f6e-4857-90be-3504d74e6dde") : object "kube-system"/"original-pull-secret" not registered
Oct 31 07:28:46.299184 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.299161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:46.299346 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.299326    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 07:28:46.299459 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.299350    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 07:28:46.299459 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.299363    2577 projected.go:196] Error preparing data for projected volume kube-api-access-5bfvr for pod openshift-network-diagnostics/network-check-target-7cl42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:46.299459 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.299424    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr podName:d4baf7ce-ca52-4d1b-a721-39ba6be4647f nodeName:}" failed. No retries permitted until 2026-10-31 07:29:02.299404993 +0000 UTC m=+34.429918330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5bfvr" (UniqueName: "kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr") pod "network-check-target-7cl42" (UID: "d4baf7ce-ca52-4d1b-a721-39ba6be4647f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 07:28:46.316042 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.316018    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:46.316123 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.316085    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:46.316203 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.316177    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:46.316203 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.316176    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:46.316344 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.316287    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:46.316409 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:46.316390    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:46.503970 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.503945    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"527a97389b9fc2bf0043a5cf27cc4cacd1c856a8339277edb42f586f12f03b0f"}
Oct 31 07:28:46.504062 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.504015    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"8b2b4b6f864ea46f8e5736185cea76fb93a1ae60a69788c6a3cf1d90dc23b42b"}
Oct 31 07:28:46.504062 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.504051    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"672b0f8a72d5135cdb7cd3840147935ce58c83a6e262f54116355b42c5616881"}
Oct 31 07:28:46.504131 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:46.504081    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"9f93851e3fd25818033fe9e1d8e9bb6b58c2a0fd3306bd235c9aeb729d19465d"}
Oct 31 07:28:47.508560 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:47.508320    2577 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="b3799c44ed5b216d01ec6c19465a11617d404dd634b544a09859625081eb85f2" exitCode=0
Oct 31 07:28:47.508560 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:47.508394    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerDied","Data":"b3799c44ed5b216d01ec6c19465a11617d404dd634b544a09859625081eb85f2"}
Oct 31 07:28:47.511688 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:47.511664    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"ef7c79b9e2e37fc50a50b94c2c27b750b6cb308f214c833107e75388e9d50308"}
Oct 31 07:28:47.511761 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:47.511747    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"929eb06c0870aad7cbc5b5bfaed1841299970124f157c35a2c8a4c8f2e8768f3"}
Oct 31 07:28:48.315150 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.315119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:48.315320 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:48.315225    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:48.315320 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.315243    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:48.315439 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:48.315353    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:48.315439 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.315386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:48.315534 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:48.315459    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:48.515847 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.515817    2577 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="56cf2da2b619257d2aaaa7a41c1c90c062e078ae1bcd0e9952641f41a95d50ed" exitCode=0
Oct 31 07:28:48.516216 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.515859    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerDied","Data":"56cf2da2b619257d2aaaa7a41c1c90c062e078ae1bcd0e9952641f41a95d50ed"}
Oct 31 07:28:48.845112 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.845045    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:48.845232 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.845149    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 07:28:48.845612 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:48.845596    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:28:49.520915 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:49.520888    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"8f30d2daa9b9fa676d50c931a9f96903484cf36315a4e37d8608bca97710c4f6"}
Oct 31 07:28:49.524285 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:49.524256    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c6r54" event={"ID":"30297c93-42e8-45c4-91b4-bc87be152069","Type":"ContainerStarted","Data":"7e3a628f1f544ff0f95f3f203479e62f52ae13e2f7e0b2ade036e9c3e646c601"}
Oct 31 07:28:49.640993 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:49.640951    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c6r54" podStartSLOduration=8.264906823 podStartE2EDuration="20.64093886s" totalImagesPullingTime="12.376032037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:49.640540037 +0000 UTC m=+21.771053395" watchObservedRunningTime="2026-10-31 07:28:49.64093886 +0000 UTC m=+21.771452218"
Oct 31 07:28:50.315155 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:50.315120    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:50.315155 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:50.315147    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:50.315351 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:50.315248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:50.315351 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:50.315253    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:50.315351 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:50.315331    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:50.315447 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:50.315404    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:51.531356 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:51.531326    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" event={"ID":"aa0b9899-ff27-48dc-85a3-e88c23f8e897","Type":"ContainerStarted","Data":"60aa2f39285b3d01bd0d4a8edf82e3b399ad4d449d671b8acccbd75809d0bf48"}
Oct 31 07:28:51.532206 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:51.531616    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:51.532206 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:51.531682    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:51.546335 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:51.546311    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:51.797874 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:51.797800    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-94t84" podStartSLOduration=7.287197661 podStartE2EDuration="22.797787105s" totalImagesPullingTime="15.510589444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:28:51.797379398 +0000 UTC m=+23.927892780" watchObservedRunningTime="2026-10-31 07:28:51.797787105 +0000 UTC m=+23.928300463"
Oct 31 07:28:52.313470 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:52.313445    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:52.313688 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:52.313449    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:52.313688 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:52.313574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:52.313688 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:52.313563    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:52.313688 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:52.313677    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:52.313806 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:52.313761    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:52.533921 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:52.533897    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:52.548301 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:52.548280    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:28:53.408538 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:53.408442    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6qkpn"]
Oct 31 07:28:53.408668 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:53.408568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:53.408668 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:53.408656    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:53.411394 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:53.411364    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cl42"]
Oct 31 07:28:53.411554 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:53.411455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:53.411624 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:53.411567    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:53.412110 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:53.412089    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w7gt6"]
Oct 31 07:28:53.412334 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:53.412313    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:53.412521 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:53.412419    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:55.313457 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:55.313429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:55.313789 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:55.313483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:55.313789 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:55.313553    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:55.313789 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:55.313556    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:55.313789 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:55.313636    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:55.313789 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:55.313702    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:57.313111 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:57.313087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:57.313550 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:57.313185    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:57.313550 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:57.313192    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:57.313550 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:57.313192    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:57.313550 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:57.313267    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:57.313550 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:57.313344    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:59.313079 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.313050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:28:59.313509 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.313088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:28:59.313509 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:59.313162    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7cl42" podUID="d4baf7ce-ca52-4d1b-a721-39ba6be4647f"
Oct 31 07:28:59.313509 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:59.313261    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7gt6" podUID="09202313-5fd7-46ab-b667-18188c9b28e7"
Oct 31 07:28:59.313509 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.313326    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:28:59.313509 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:59.313404    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6qkpn" podUID="65d23093-9f6e-4857-90be-3504d74e6dde"
Oct 31 07:28:59.590076 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.590057    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeReady"
Oct 31 07:28:59.590200 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.590151    2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 07:28:59.700812 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.700786    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qdpz4"]
Oct 31 07:28:59.745452 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.745429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:28:59.750137 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.750112    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mqsmg"]
Oct 31 07:28:59.763725 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.763705    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.801797 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.801767    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 07:28:59.801797 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.801793    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 07:28:59.802117 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.802103    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 07:28:59.802186 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.802118    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-m5kd5\""
Oct 31 07:28:59.811773 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.811743    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qdpz4"]
Oct 31 07:28:59.814813 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.814755    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mqsmg"]
Oct 31 07:28:59.897983 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.897930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twfwt\" (UniqueName: \"kubernetes.io/projected/241ac3ab-0782-4990-87b8-d0199fc34945-kube-api-access-twfwt\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.897983 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.897960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.898130 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.897988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/241ac3ab-0782-4990-87b8-d0199fc34945-config-volume\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.898130 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.898075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831a61ee-1d95-484a-a379-434ea9284887-cert\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:28:59.898207 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.898189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/241ac3ab-0782-4990-87b8-d0199fc34945-tmp-dir\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.898251 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.898215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdvq9\" (UniqueName: \"kubernetes.io/projected/831a61ee-1d95-484a-a379-434ea9284887-kube-api-access-rdvq9\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:28:59.905412 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.905390    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 07:28:59.905535 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.905521    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 07:28:59.905597 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.905581    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xb4h7\""
Oct 31 07:28:59.998614 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.998592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdvq9\" (UniqueName: \"kubernetes.io/projected/831a61ee-1d95-484a-a379-434ea9284887-kube-api-access-rdvq9\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:28:59.998712 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.998631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twfwt\" (UniqueName: \"kubernetes.io/projected/241ac3ab-0782-4990-87b8-d0199fc34945-kube-api-access-twfwt\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.998712 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.998651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.998712 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.998671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/241ac3ab-0782-4990-87b8-d0199fc34945-config-volume\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.998858 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:59.998763    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:28:59.998858 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:28:59.998812    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:00.498798199 +0000 UTC m=+32.629311569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:28:59.998858 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.998852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831a61ee-1d95-484a-a379-434ea9284887-cert\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:28:59.999006 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.998889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/241ac3ab-0782-4990-87b8-d0199fc34945-tmp-dir\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.999202 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.999187    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/241ac3ab-0782-4990-87b8-d0199fc34945-tmp-dir\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:28:59.999450 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:28:59.999431    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/241ac3ab-0782-4990-87b8-d0199fc34945-config-volume\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:00.002373 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:00.002350    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831a61ee-1d95-484a-a379-434ea9284887-cert\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:29:00.054842 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:00.054817    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twfwt\" (UniqueName: \"kubernetes.io/projected/241ac3ab-0782-4990-87b8-d0199fc34945-kube-api-access-twfwt\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:00.055305 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:00.055290    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdvq9\" (UniqueName: \"kubernetes.io/projected/831a61ee-1d95-484a-a379-434ea9284887-kube-api-access-rdvq9\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:29:00.064014 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:00.063995    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:29:00.497926 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:00.497862    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qdpz4"]
Oct 31 07:29:00.500691 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:29:00.500669    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831a61ee_1d95_484a_a379_434ea9284887.slice/crio-fdbf44e3a2c45a72857ddb26eb6a8c0ca20c0a73c8b05a6df1a645177289a241 WatchSource:0}: Error finding container fdbf44e3a2c45a72857ddb26eb6a8c0ca20c0a73c8b05a6df1a645177289a241: Status 404 returned error can't find the container with id fdbf44e3a2c45a72857ddb26eb6a8c0ca20c0a73c8b05a6df1a645177289a241
Oct 31 07:29:00.502533 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:00.502516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:00.502657 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:00.502644    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:00.502701 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:00.502691    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:01.502679869 +0000 UTC m=+33.633193219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:00.548783 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:00.548762    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qdpz4" event={"ID":"831a61ee-1d95-484a-a379-434ea9284887","Type":"ContainerStarted","Data":"fdbf44e3a2c45a72857ddb26eb6a8c0ca20c0a73c8b05a6df1a645177289a241"}
Oct 31 07:29:01.312900 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.312871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:01.313061 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.312879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:01.313061 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.313008    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:01.364719 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.364699    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 07:29:01.364819 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.364718    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bpwql\""
Oct 31 07:29:01.364819 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.364698    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 07:29:01.414254 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.414233    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 07:29:01.414365 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.414302    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-5qqnh\""
Oct 31 07:29:01.414365 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.414315    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-crxcw\""
Oct 31 07:29:01.414485 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.414417    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 07:29:01.508933 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:01.508907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:01.509272 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:01.509030    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:01.509272 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:01.509098    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:03.509079998 +0000 UTC m=+35.639593351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:02.113531 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.113494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:02.113715 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:02.113621    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 07:29:02.113715 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:02.113694    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:34.11367569 +0000 UTC m=+66.244189027 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : secret "metrics-daemon-secret" not found
Oct 31 07:29:02.214083 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.213977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:02.216564 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.216538    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:02.280762 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.280733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:02.314851 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.314823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:02.317580 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.317560    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:02.600435 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.600400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:02.703590 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.703489    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6qkpn"]
Oct 31 07:29:02.705827 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:29:02.705803    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d23093_9f6e_4857_90be_3504d74e6dde.slice/crio-d4da9cf5d72bafa8b1a03c925db6c9ca1f7d225a12a0f76d714d6fce90f73a68 WatchSource:0}: Error finding container d4da9cf5d72bafa8b1a03c925db6c9ca1f7d225a12a0f76d714d6fce90f73a68: Status 404 returned error can't find the container with id d4da9cf5d72bafa8b1a03c925db6c9ca1f7d225a12a0f76d714d6fce90f73a68
Oct 31 07:29:02.854939 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.854908    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5gj7z"]
Oct 31 07:29:02.909093 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.909067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:02.950850 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.950739    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5gj7z"]
Oct 31 07:29:02.995122 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:29:02.995085    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4baf7ce_ca52_4d1b_a721_39ba6be4647f.slice/crio-fbeb4283fa1a0d5e72b7bc850c83e42222985c7bc1e811ec4e6e836583a8ac65 WatchSource:0}: Error finding container fbeb4283fa1a0d5e72b7bc850c83e42222985c7bc1e811ec4e6e836583a8ac65: Status 404 returned error can't find the container with id fbeb4283fa1a0d5e72b7bc850c83e42222985c7bc1e811ec4e6e836583a8ac65
Oct 31 07:29:02.999999 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:02.999974    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7cl42"]
Oct 31 07:29:03.017411 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.017389    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 07:29:03.017626 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.017607    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g75wk\""
Oct 31 07:29:03.021865 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.021842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-data-volume\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.021952 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.021915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.022048 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.022027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-crio-socket\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.022107 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.022082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.022180 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.022160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvmbp\" (UniqueName: \"kubernetes.io/projected/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-api-access-kvmbp\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.040749 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.040728    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 07:29:03.040867 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.040753    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 07:29:03.047432 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.047417    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 07:29:03.102609 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.102581    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-x4b5m-ps6h6"]
Oct 31 07:29:03.123343 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.123308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.123487 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.123354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-crio-socket\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.123487 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.123393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.123487 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.123418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvmbp\" (UniqueName: \"kubernetes.io/projected/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-api-access-kvmbp\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.123487 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.123444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-data-volume\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.123686 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.123489    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:03.123686 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.123539    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 07:29:03.123686 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.123569    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:03.623549078 +0000 UTC m=+35.754062431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:03.123686 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.123607    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:03.623591174 +0000 UTC m=+35.754104527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : configmap "kube-rbac-proxy" not found
Oct 31 07:29:03.123686 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.123648    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-crio-socket\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.123896 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.123816    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-data-volume\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.134557 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.134540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.161675 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.161653    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-x4b5m-ps6h6"]
Oct 31 07:29:03.180022 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.179994    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvmbp\" (UniqueName: \"kubernetes.io/projected/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-api-access-kvmbp\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.218435 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.218373    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pkrx7\""
Oct 31 07:29:03.224710 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.224690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktj2q\" (UniqueName: \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.224806 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.224726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.224868 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.224849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.224921 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.224897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.241030 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.241009    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 07:29:03.291448 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.291419    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 07:29:03.325279 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.325254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.325402 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.325305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.325402 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.325336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.325402 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.325394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktj2q\" (UniqueName: \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.325766 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.325745    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.325874 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.325858    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.326070 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.326053    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.381650 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.381624    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktj2q\" (UniqueName: \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.463343 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.463319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:03.541670 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.541414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:03.541670 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.541604    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:03.541845 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.541682    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:07.541666444 +0000 UTC m=+39.672179780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:03.557330 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.557303    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cl42" event={"ID":"d4baf7ce-ca52-4d1b-a721-39ba6be4647f","Type":"ContainerStarted","Data":"fbeb4283fa1a0d5e72b7bc850c83e42222985c7bc1e811ec4e6e836583a8ac65"}
Oct 31 07:29:03.558324 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.558304    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6qkpn" event={"ID":"65d23093-9f6e-4857-90be-3504d74e6dde","Type":"ContainerStarted","Data":"d4da9cf5d72bafa8b1a03c925db6c9ca1f7d225a12a0f76d714d6fce90f73a68"}
Oct 31 07:29:03.642341 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.642316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.642642 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.642356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:03.642642 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.642417    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 07:29:03.642642 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.642447    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:03.642642 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.642480    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:04.642452195 +0000 UTC m=+36.772965544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : configmap "kube-rbac-proxy" not found
Oct 31 07:29:03.642642 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:03.642517    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:04.642502608 +0000 UTC m=+36.773015943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:03.759988 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.759960    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"]
Oct 31 07:29:03.797870 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.797845    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"
Oct 31 07:29:03.818977 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.818956    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"]
Oct 31 07:29:03.847988 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.847953    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-x4b5m-ps6h6"]
Oct 31 07:29:03.851876 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:29:03.851848    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod529a9ab9_cc1d_42a4_b34a_4cc09a38b81c.slice/crio-8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7 WatchSource:0}: Error finding container 8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7: Status 404 returned error can't find the container with id 8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7
Oct 31 07:29:03.853808 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.853718    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6g5bx\""
Oct 31 07:29:03.937619 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.937381    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:03.937619 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.937534    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 07:29:03.945064 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:03.945034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jndb\" (UniqueName: \"kubernetes.io/projected/51974c85-319d-4074-b109-3927c8b0e147-kube-api-access-7jndb\") pod \"migrator-66df5d496b-f9w6k\" (UID: \"51974c85-319d-4074-b109-3927c8b0e147\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"
Oct 31 07:29:04.045758 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.045724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jndb\" (UniqueName: \"kubernetes.io/projected/51974c85-319d-4074-b109-3927c8b0e147-kube-api-access-7jndb\") pod \"migrator-66df5d496b-f9w6k\" (UID: \"51974c85-319d-4074-b109-3927c8b0e147\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"
Oct 31 07:29:04.101814 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.101786    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jndb\" (UniqueName: \"kubernetes.io/projected/51974c85-319d-4074-b109-3927c8b0e147-kube-api-access-7jndb\") pod \"migrator-66df5d496b-f9w6k\" (UID: \"51974c85-319d-4074-b109-3927c8b0e147\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"
Oct 31 07:29:04.122641 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.122613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"
Oct 31 07:29:04.437680 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.436701    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-xrxjs"]
Oct 31 07:29:04.461424 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.461397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.498093 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.498060    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-xrxjs"]
Oct 31 07:29:04.512674 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.512647    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"]
Oct 31 07:29:04.513377 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.513351    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 31 07:29:04.513503 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.513439    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 31 07:29:04.513591 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.513555    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-845kx\""
Oct 31 07:29:04.513591 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.513569    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 31 07:29:04.513667 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.513636    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 31 07:29:04.522969 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:29:04.522942    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51974c85_319d_4074_b109_3927c8b0e147.slice/crio-368da24d56f70bc8f4fe3fc6174f63d16567232e3a78b7dcee350002645fafbc WatchSource:0}: Error finding container 368da24d56f70bc8f4fe3fc6174f63d16567232e3a78b7dcee350002645fafbc: Status 404 returned error can't find the container with id 368da24d56f70bc8f4fe3fc6174f63d16567232e3a78b7dcee350002645fafbc
Oct 31 07:29:04.562963 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.562931    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k" event={"ID":"51974c85-319d-4074-b109-3927c8b0e147","Type":"ContainerStarted","Data":"368da24d56f70bc8f4fe3fc6174f63d16567232e3a78b7dcee350002645fafbc"}
Oct 31 07:29:04.565349 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.564854    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qdpz4" event={"ID":"831a61ee-1d95-484a-a379-434ea9284887","Type":"ContainerStarted","Data":"e78fa8f02f44350af03fb2132ceff31d69ebe92cf102714c93eaf077b0b5f9b0"}
Oct 31 07:29:04.566748 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.566722    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6" event={"ID":"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c","Type":"ContainerStarted","Data":"8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7"}
Oct 31 07:29:04.650731 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.650698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.650741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq7bf\" (UniqueName: \"kubernetes.io/projected/cafe16db-7076-474d-826c-f908e4f4a71f-kube-api-access-xq7bf\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.650767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cafe16db-7076-474d-826c-f908e4f4a71f-signing-key\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.650821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:04.650845    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:04.650907    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:04.650914    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:06.650895018 +0000 UTC m=+38.781408371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:04.650964    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:06.650952737 +0000 UTC m=+38.781466072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : configmap "kube-rbac-proxy" not found
Oct 31 07:29:04.651131 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.650984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cafe16db-7076-474d-826c-f908e4f4a71f-signing-cabundle\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.684526 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.684454    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qdpz4" podStartSLOduration=2.47418519 podStartE2EDuration="5.684438108s" totalImagesPullingTime="3.210252918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:04.684107358 +0000 UTC m=+36.814620718" watchObservedRunningTime="2026-10-31 07:29:04.684438108 +0000 UTC m=+36.814951468"
Oct 31 07:29:04.751582 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.751548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cafe16db-7076-474d-826c-f908e4f4a71f-signing-cabundle\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.751719 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.751644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq7bf\" (UniqueName: \"kubernetes.io/projected/cafe16db-7076-474d-826c-f908e4f4a71f-kube-api-access-xq7bf\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.751719 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.751672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cafe16db-7076-474d-826c-f908e4f4a71f-signing-key\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.757602 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.756318    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cafe16db-7076-474d-826c-f908e4f4a71f-signing-cabundle\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.757978 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.757953    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cafe16db-7076-474d-826c-f908e4f4a71f-signing-key\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:04.812674 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:04.812621    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq7bf\" (UniqueName: \"kubernetes.io/projected/cafe16db-7076-474d-826c-f908e4f4a71f-kube-api-access-xq7bf\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:05.085744 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:05.085708    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:05.485095 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:05.484970    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-xrxjs"]
Oct 31 07:29:05.496775 ip-10-0-133-50 kubenswrapper[2577]: W1031 07:29:05.496708    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcafe16db_7076_474d_826c_f908e4f4a71f.slice/crio-0910090c884c482f05e65f42c0d4d6e636bc5f27fb29799726adc95883b991a9 WatchSource:0}: Error finding container 0910090c884c482f05e65f42c0d4d6e636bc5f27fb29799726adc95883b991a9: Status 404 returned error can't find the container with id 0910090c884c482f05e65f42c0d4d6e636bc5f27fb29799726adc95883b991a9
Oct 31 07:29:05.573142 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:05.573108    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs" event={"ID":"cafe16db-7076-474d-826c-f908e4f4a71f","Type":"ContainerStarted","Data":"0910090c884c482f05e65f42c0d4d6e636bc5f27fb29799726adc95883b991a9"}
Oct 31 07:29:06.669492 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:06.669439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:06.669971 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:06.669543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:06.669971 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:06.669640    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:06.669971 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:06.669698    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:10.669679026 +0000 UTC m=+42.800192363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:06.670096 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:06.670014    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:07.575943 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:07.575903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:07.576197 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:07.576170    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:07.576260 ip-10-0-133-50 kubenswrapper[2577]: E1031 07:29:07.576244    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:15.576225149 +0000 UTC m=+47.706738488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:07.582558 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:07.582522    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6" event={"ID":"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c","Type":"ContainerStarted","Data":"7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60"}
Oct 31 07:29:07.584744 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:07.584715    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k" event={"ID":"51974c85-319d-4074-b109-3927c8b0e147","Type":"ContainerStarted","Data":"47824c6f958e844110b9de40c6c1b3c92c2cfe9dad8008468704618b856462f8"}
Oct 31 07:29:07.718314 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:07.718264    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6" podStartSLOduration=1.154251662 podStartE2EDuration="4.718247375s" totalImagesPullingTime="3.563995713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:07.716911764 +0000 UTC m=+39.847425122" watchObservedRunningTime="2026-10-31 07:29:07.718247375 +0000 UTC m=+39.848760732"
Oct 31 07:29:07.833778 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:07.833714    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k" podStartSLOduration=2.341276922 podStartE2EDuration="4.833694035s" totalImagesPullingTime="2.492417113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:07.833633799 +0000 UTC m=+39.964147158" watchObservedRunningTime="2026-10-31 07:29:07.833694035 +0000 UTC m=+39.964207395"
Oct 31 07:29:08.588913 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:08.588869    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7cl42" event={"ID":"d4baf7ce-ca52-4d1b-a721-39ba6be4647f","Type":"ContainerStarted","Data":"891353b5b4568356810abd9144b8a187f3605d391715cdd492949c13ba460b7a"}
Oct 31 07:29:08.589177 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:08.589158    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:08.701602 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:08.701550    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7cl42" podStartSLOduration=35.036468155 podStartE2EDuration="39.701535601s" totalImagesPullingTime="4.665067446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:08.701003815 +0000 UTC m=+40.831517173" watchObservedRunningTime="2026-10-31 07:29:08.701535601 +0000 UTC m=+40.832048958"
Oct 31 07:29:09.437299 ip-10-0-133-50 kubenswrapper[2577]: I1031 07:29:09.437097    2577 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:29:09.437247 ip-10-0-133-50 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 07:29:09.459976 ip-10-0-133-50 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 07:29:09.460226 ip-10-0-133-50 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 07:29:09.460292 ip-10-0-133-50 systemd[1]: kubelet.service: Consumed 2.318s CPU time.
Oct 31 07:29:09.469063 ip-10-0-133-50 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 07:29:09.545206 ip-10-0-133-50 kubenswrapper[4532]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:29:09.545206 ip-10-0-133-50 kubenswrapper[4532]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 07:29:09.545206 ip-10-0-133-50 kubenswrapper[4532]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:29:09.545206 ip-10-0-133-50 kubenswrapper[4532]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 07:29:09.547264 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547187    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.547264 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547262    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.547264 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547266    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.547264 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547269    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547273    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547276    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547279    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547282    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547286    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547288    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547291    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547294    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547297    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547300    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547303    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547305    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547308    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547317    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547322    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547326    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547330    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547334    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547338    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.547427 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547342    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547347    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547350    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547353    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547356    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547359    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547362    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547364    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547367    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547370    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547372    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547375    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547378    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547380    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547383    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547386    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547388    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547391    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547393    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547397    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.547935 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547400    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547402    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547405    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547407    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547409    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547412    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547415    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547418    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547420    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547423    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547425    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547428    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547431    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547434    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547437    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547440    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547443    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547446    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547449    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547451    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.548477 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547454    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547456    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547471    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547474    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547477    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547480    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547482    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547485    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547487    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547490    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547492    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547495    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547497    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547500    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547502    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547505    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547508    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547512    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547517    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.548980 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547521    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547524    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547526    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547529    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547532    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547536    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547540    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547543    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547545    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547548    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547550    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547553    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547556    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547558    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547561    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547563    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547566    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547568    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547572    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.549669 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547576    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547580    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547583    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547586    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.547589    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548352    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548359    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548362    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548365    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548368    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548371    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548373    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548376    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548378    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548382    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548385    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548388    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548390    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548393    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548396    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.550380 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548399    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548402    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548404    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548407    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548410    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548412    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548415    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548418    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548420    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548423    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548425    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548428    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548431    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548434    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548437    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548439    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548442    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548446    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548450    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.550911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548453    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548456    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548473    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548476    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548480    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548483    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548486    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548489    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548492    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548495    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548498    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548501    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548503    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548506    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548508    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548511    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548513    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548516    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548518    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548521    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.551392 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548523    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548526    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548529    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548531    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548534    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548536    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548539    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548541    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548544    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548546    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548549    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548552    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548554    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548558    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548560    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548563    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548566    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548568    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548571    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548573    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.551905 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548576    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548579    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548582    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548584    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548587    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548589    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548592    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548595    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548597    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548600    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548602    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548605    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548607    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548610    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548612    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548615    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548617    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548619    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548623    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.552390 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548627    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548630    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548633    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548637    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548640    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548642    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548645    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548648    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548651    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548654    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548656    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548659    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.548661    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549170    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549177    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549181    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549185    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549188    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549191    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549194    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.552861 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549197    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549199    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549202    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549205    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549207    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549210    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549212    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549215    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549218    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549220    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549223    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549226    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549228    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549231    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549233    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549236    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549239    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549241    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549244    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549247    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.553353 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549249    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549252    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549254    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549257    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549259    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549262    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549264    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549267    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549271    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549275    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549278    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549280    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549284    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549286    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549289    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549291    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549294    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549297    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549300    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.553852 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549303    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549305    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549308    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549310    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549313    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549315    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549318    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549320    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549323    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549325    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549327    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549330    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549333    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549335    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549338    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549340    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549342    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549345    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549347    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549350    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.554355 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549352    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549355    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549358    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549361    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549364    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549366    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549369    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549371    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549377    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549379    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549382    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549385    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549387    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549390    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549393    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549395    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549398    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549401    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549403    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549406    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.554855 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549408    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549411    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549414    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549416    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549419    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549421    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549424    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549426    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549429    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549431    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549434    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549436    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549439    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549441    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549445    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549449    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549452    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549454    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549457    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.549473    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549572    4532 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 07:29:09.555472 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549582    4532 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549591    4532 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549598    4532 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549606    4532 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549610    4532 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549614    4532 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549618    4532 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549622    4532 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549625    4532 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549628    4532 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549631    4532 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549635    4532 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549638    4532 flags.go:64] FLAG: --cgroup-root=""
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549641    4532 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549644    4532 flags.go:64] FLAG: --client-ca-file=""
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549647    4532 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549650    4532 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549679    4532 flags.go:64] FLAG: --cluster-domain=""
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549682    4532 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549686    4532 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549690    4532 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549693    4532 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549698    4532 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549701    4532 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 07:29:09.555994 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549704    4532 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549707    4532 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549711    4532 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549714    4532 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549717    4532 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549721    4532 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549724    4532 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549730    4532 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549733    4532 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549736    4532 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549739    4532 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549742    4532 flags.go:64] FLAG: --enable-server="true"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549745    4532 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549749    4532 flags.go:64] FLAG: --event-burst="100"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549752    4532 flags.go:64] FLAG: --event-qps="50"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549755    4532 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549758    4532 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549761    4532 flags.go:64] FLAG: --eviction-hard=""
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549766    4532 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549769    4532 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549772    4532 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549775    4532 flags.go:64] FLAG: --eviction-soft=""
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549778    4532 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549782    4532 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549785    4532 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 07:29:09.556597 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549788    4532 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549791    4532 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549794    4532 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549797    4532 flags.go:64] FLAG: --feature-gates=""
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549801    4532 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549805    4532 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549808    4532 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549811    4532 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549814    4532 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549818    4532 flags.go:64] FLAG: --help="false"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549821    4532 flags.go:64] FLAG: --hostname-override="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549824    4532 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549827    4532 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549830    4532 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549835    4532 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549839    4532 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549842    4532 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549845    4532 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549848    4532 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549851    4532 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549854    4532 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549857    4532 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549860    4532 flags.go:64] FLAG: --kube-reserved=""
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549863    4532 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 07:29:09.557290 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549866    4532 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549869    4532 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549871    4532 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549874    4532 flags.go:64] FLAG: --lock-file=""
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549880    4532 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549883    4532 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549886    4532 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549892    4532 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549895    4532 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549898    4532 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549901    4532 flags.go:64] FLAG: --logging-format="text"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549904    4532 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549907    4532 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549910    4532 flags.go:64] FLAG: --manifest-url=""
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549913    4532 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549918    4532 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549921    4532 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549925    4532 flags.go:64] FLAG: --max-pods="110"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549928    4532 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549931    4532 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549934    4532 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549937    4532 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549941    4532 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549944    4532 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549947    4532 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 07:29:09.558322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549954    4532 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549958    4532 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549961    4532 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549964    4532 flags.go:64] FLAG: --pod-cidr=""
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549967    4532 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549970    4532 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549973    4532 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549976    4532 flags.go:64] FLAG: --port="10250"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549979    4532 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549982    4532 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0d70d3c41c368c420"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549985    4532 flags.go:64] FLAG: --qos-reserved=""
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549990    4532 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549993    4532 flags.go:64] FLAG: --register-node="true"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.549996    4532 flags.go:64] FLAG: --register-with-taints=""
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550000    4532 flags.go:64] FLAG: --registry-burst="10"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550003    4532 flags.go:64] FLAG: --registry-qps="5"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550005    4532 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550008    4532 flags.go:64] FLAG: --reserved-memory=""
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550012    4532 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550015    4532 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550018    4532 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550022    4532 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550025    4532 flags.go:64] FLAG: --runonce="false"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550028    4532 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550031    4532 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550035    4532 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550038    4532 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 07:29:09.559294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550041    4532 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550044    4532 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550048    4532 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550051    4532 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550054    4532 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550057    4532 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550060    4532 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550063    4532 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550066    4532 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550069    4532 flags.go:64] FLAG: --system-cgroups=""
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550072    4532 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550077    4532 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550080    4532 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550083    4532 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550087    4532 flags.go:64] FLAG: --tls-min-version=""
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550090    4532 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550095    4532 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550098    4532 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550101    4532 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550104    4532 flags.go:64] FLAG: --v="2"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550108    4532 flags.go:64] FLAG: --version="false"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550112    4532 flags.go:64] FLAG: --vmodule=""
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550116    4532 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550119    4532 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550248    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.560503 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550253    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550259    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550264    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550268    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550272    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550276    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550279    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550282    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550286    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550292    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550295    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550297    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550300    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550303    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550305    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550308    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550311    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550313    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550316    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.561357 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550319    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550322    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550324    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550327    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550331    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550334    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550337    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550342    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550346    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550351    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550355    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550359    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550362    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550365    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550368    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550371    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550373    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550377    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550380    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.562060 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550383    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550387    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550391    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550399    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550403    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550406    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550408    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550411    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550414    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550416    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550420    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550424    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550428    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550432    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550436    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550441    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550444    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550448    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550451    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550455    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.562648 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550473    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550477    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550480    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550483    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550486    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550490    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550492    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550495    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550498    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550500    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550503    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550508    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550512    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550516    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550520    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550525    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550531    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550535    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550539    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550543    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.563152 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550547    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550551    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550555    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550559    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550563    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550567    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550571    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550578    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550585    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550592    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550597    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550601    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550606    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550609    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550613    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550617    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550622    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550626    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550630    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550634    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.563793 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550639    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550643    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550645    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550648    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550651    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550653    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550656    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.550688    4532 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550723    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550728    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550732    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550735    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550738    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550741    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550744    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550747    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.564328 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550749    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550752    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550755    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550757    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550760    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550763    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550768    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550770    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550773    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550775    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550778    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550781    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550784    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550787    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550789    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550792    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550794    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550797    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550800    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550802    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.564913 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550805    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550807    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550810    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550812    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550815    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550817    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550821    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550823    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550826    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550828    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550831    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550833    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550836    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550839    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550841    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550844    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550847    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550849    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550853    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.565429 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550856    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550858    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550861    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550863    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550866    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550869    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550872    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550874    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550877    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550880    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550882    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550885    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550887    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550890    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550892    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550895    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550898    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550900    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550902    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.565896 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550905    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550908    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550911    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550913    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550916    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550919    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550922    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550924    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550927    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550929    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550932    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550936    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550942    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550945    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550948    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550950    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550954    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550957    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550959    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550962    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.566360 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550965    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550967    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550970    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550972    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550975    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550977    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550980    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550983    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550986    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550988    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550991    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550993    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550996    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.550998    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.551001    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.551004    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.551007    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.551009    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.551012    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.566991 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.551017    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.554546    4532 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.554561    4532 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.554578    4532 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.554582    4532 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554636    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554641    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554645    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554648    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554652    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554655    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554658    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554661    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554664    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554667    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554669    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554672    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554676    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554680    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.567475 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554683    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554685    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554688    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554692    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554695    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554698    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554701    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554703    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554706    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554709    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554711    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554714    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554717    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554720    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554722    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554725    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554728    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554731    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554734    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554736    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.567955 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554739    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554742    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554744    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554747    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554751    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554754    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554757    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554760    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554762    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554765    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554767    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554770    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554773    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554775    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554777    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554781    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554784    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554787    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554790    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.568445 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554792    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554795    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554798    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554800    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554803    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554805    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554808    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554810    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554813    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554815    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554818    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554820    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554823    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554825    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554828    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554831    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554833    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554836    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554838    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554841    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.568962 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554843    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554846    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554848    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554851    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554853    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554856    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554859    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554862    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554865    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554868    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554870    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554873    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554876    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554879    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554881    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554883    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554886    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554889    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554891    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554894    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.569446 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554896    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554899    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554902    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554905    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554907    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554910    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554913    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554915    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554918    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554921    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554923    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554926    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.554928    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.554955    4532 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555095    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.569938 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555100    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555103    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555105    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555108    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555111    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555114    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555118    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555121    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555124    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555127    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555131    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555133    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555136    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555138    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555142    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555144    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555147    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555150    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555152    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.570337 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555155    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555157    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555160    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555162    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555165    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555167    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555170    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555172    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555175    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555178    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555180    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555183    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555185    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555188    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555190    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555193    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555196    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555198    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555201    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555203    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.570815 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555206    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555208    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555211    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555213    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555217    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555220    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555222    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555225    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555227    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555230    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555232    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555236    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555239    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555242    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555245    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555247    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555250    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555252    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555255    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.571301 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555258    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555260    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555263    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555265    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555268    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555270    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555273    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555276    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555278    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555281    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555283    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555286    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555288    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555291    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555294    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555296    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555299    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555301    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555304    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.571808 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555307    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555309    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555312    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555314    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555317    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555319    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555322    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555324    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555326    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555329    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555332    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555334    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555337    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555340    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555342    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555345    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555347    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555350    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555352    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555355    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.572278 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555358    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555360    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555363    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555365    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555368    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555370    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555373    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555375    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.555398    4532 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555426    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555429    4532 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555433    4532 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555436    4532 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555439    4532 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555442    4532 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 07:29:09.572870 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555446    4532 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555448    4532 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555451    4532 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555453    4532 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555456    4532 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555459    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555473    4532 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555476    4532 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555479    4532 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555482    4532 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555485    4532 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555488    4532 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555490    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555493    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555496    4532 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555498    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555504    4532 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555506    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555509    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 07:29:09.573290 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555511    4532 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555514    4532 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555517    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555519    4532 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555522    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555524    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555527    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555529    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555532    4532 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555535    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555537    4532 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555540    4532 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555543    4532 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555548    4532 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555550    4532 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555553    4532 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555555    4532 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555558    4532 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555561    4532 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555564    4532 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555567    4532 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 07:29:09.574013 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555569    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555572    4532 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555575    4532 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555577    4532 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555580    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555583    4532 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555585    4532 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555588    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555592    4532 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555595    4532 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555597    4532 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555600    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555602    4532 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555605    4532 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555607    4532 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555610    4532 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555613    4532 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555615    4532 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555618    4532 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 07:29:09.574911 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555621    4532 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555623    4532 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555627    4532 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555631    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555635    4532 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555640    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555644    4532 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555646    4532 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555649    4532 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555652    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555654    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555657    4532 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555659    4532 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555662    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555665    4532 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555668    4532 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555670    4532 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555673    4532 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555675    4532 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 07:29:09.576785 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555678    4532 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555680    4532 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555684    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555687    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555689    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555692    4532 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555695    4532 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555697    4532 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555700    4532 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555702    4532 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555705    4532 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555707    4532 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555710    4532 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555712    4532 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555715    4532 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555717    4532 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555720    4532 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555722    4532 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555726    4532 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555729    4532 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 07:29:09.577515 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555733    4532 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:09.555736    4532 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.555784    4532 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.556741    4532 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.556790    4532 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.556963    4532 server.go:1014] "Starting client certificate rotation"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.557098    4532 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.557332    4532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 07:23:29 +0000 UTC" deadline="2028-06-18 23:41:54.32642593 +0000 UTC"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.557376    4532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14320h12m44.769053181s"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.558186    4532 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.560294    4532 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.561878    4532 log.go:25] "Validated CRI v1 runtime API"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.563444    4532 log.go:25] "Validated CRI v1 image API"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.564242    4532 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 07:29:09.578050 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.565776    4532 fs.go:135] Filesystem UUIDs: map[02ef2c36-5ea7-4330-a1dc-2ee4996e5206:/dev/nvme0n1p4 5e685bf5-adf7-413d-bbe3-6a1c01366601:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 07:29:09.578406 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.565797    4532 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0910090c884c482f05e65f42c0d4d6e636bc5f27fb29799726adc95883b991a9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0910090c884c482f05e65f42c0d4d6e636bc5f27fb29799726adc95883b991a9/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0fa0de5fc59ed79c13caa15a341e48b93450e339420b8151da16d13f476bee10/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0fa0de5fc59ed79c13caa15a341e48b93450e339420b8151da16d13f476bee10/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/23a52a8681039cf98cc7053c4639885ecbdfbb6a0cfd9c34446efc7db25d74db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/23a52a8681039cf98cc7053c4639885ecbdfbb6a0cfd9c34446efc7db25d74db/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/368da24d56f70bc8f4fe3fc6174f63d16567232e3a78b7dcee350002645fafbc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/368da24d56f70bc8f4fe3fc6174f63d16567232e3a78b7dcee350002645fafbc/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/395e5c0f873407f0f6afad75c577b5435b5d9c19f9be4048b567325b58fb447a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/395e5c0f873407f0f6afad75c577b5435b5d9c19f9be4048b567325b58fb447a/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/583c98cf4ca216c2bcb77d9e45d3d6cbeacbf83325e0af838793af9a4fe02375/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/583c98cf4ca216c2bcb77d9e45d3d6cbeacbf83325e0af838793af9a4fe02375/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8dbd7dfc6c19def94c43ca469fec4670c8c38ffa3454b40a67cc1cc84f07271f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8dbd7dfc6c19def94c43ca469fec4670c8c38ffa3454b40a67cc1cc84f07271f/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c19671c9797d3022d0ec1596954989a36cd362ea11f6d2231ad3e94b7608aefa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c19671c9797d3022d0ec1596954989a36cd362ea11f6d2231ad3e94b7608aefa/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cf1faebd06b43da90ccb06bfd0490bf51eca7690822fea73336a7e1173c32523/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cf1faebd06b43da90ccb06bfd0490bf51eca7690822fea73336a7e1173c32523/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d4da9cf5d72bafa8b1a03c925db6c9ca1f7d225a12a0f76d714d6fce90f73a68/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d4da9cf5d72bafa8b1a03c925db6c9ca1f7d225a12a0f76d714d6fce90f73a68/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f3b4918a8db6722bff832095e27346be39aa09fc3f6e3ad57d40d5a9c198c614/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f3b4918a8db6722bff832095e27346be39aa09fc3f6e3ad57d40d5a9c198c614/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fbeb4283fa1a0d5e72b7bc850c83e42222985c7bc1e811ec4e6e836583a8ac65/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fbeb4283fa1a0d5e72b7bc850c83e42222985c7bc1e811ec4e6e836583a8ac65/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fdbf44e3a2c45a72857ddb26eb6a8c0ca20c0a73c8b05a6df1a645177289a241/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fdbf44e3a2c45a72857ddb26eb6a8c0ca20c0a73c8b05a6df1a645177289a241/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09202313-5fd7-46ab-b667-18188c9b28e7/volumes/kubernetes.io~projected/kube-api-access-nprzm:{mountpoint:/var/lib/kubelet/pods/09202313-5fd7-46ab-b667-18188c9b28e7/volumes/kubernetes.io~projected/kube-api-access-nprzm major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/241ac3ab-0782-4990-87b8-d0199fc34945/volumes/kubernetes.io~projected/kube-api-access-twfwt:{mountpoint:/var/lib/kubelet/pods/241ac3ab-0782-4990-87b8-d0199fc34945/volumes/kubernetes.io~projected/kube-api-access-twfwt major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30297c93-42e8-45c4-91b4-bc87be152069/volumes/kubernetes.io~projected/kube-api-access-7qxq2:{mountpoint:/var/lib/kubelet/pods/30297c93-42e8-45c4-91b4-bc87be152069/volumes/kubernetes.io~projected/kube-api-access-7qxq2 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46ca8ac6-dfed-461e-90ed-17556a2e5272/volumes/kubernetes.io~projected/kube-api-access-99wxd:{mountpoint:/var/lib/kubelet/pods/46ca8ac6-dfed-461e-90ed-17556a2e5272/volumes/kubernetes.io~projected/kube-api-access-99wxd major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51974c85-319d-4074-b109-3927c8b0e147/volumes/kubernetes.io~projected/kube-api-access-7jndb:{mountpoint:/var/lib/kubelet/pods/51974c85-319d-4074-b109-3927c8b0e147/volumes/kubernetes.io~projected/kube-api-access-7jndb major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c/volumes/kubernetes.io~projected/kube-api-access-ktj2q:{mountpoint:/var/lib/kubelet/pods/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c/volumes/kubernetes.io~projected/kube-api-access-ktj2q major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65d23093-9f6e-4857-90be-3504d74e6dde/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/65d23093-9f6e-4857-90be-3504d74e6dde/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~empty-dir/tmp major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~projected/kube-api-access-nghgg:{mountpoint:/var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~projected/kube-api-access-nghgg major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/831a61ee-1d95-484a-a379-434ea9284887/volumes/kubernetes.io~projected/kube-api-access-rdvq9:{mountpoint:/var/lib/kubelet/pods/831a61ee-1d95-484a-a379-434ea9284887/volumes/kubernetes.io~projected/kube-api-access-rdvq9 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/831a61ee-1d95-484a-a379-434ea9284887/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/831a61ee-1d95-484a-a379-434ea9284887/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/98e9b442-22a5-4329-9a1f-872e81ff6505/volumes/kubernetes.io~projected/kube-api-access-qb6tz:{mountpoint:/var/lib/kubelet/pods/98e9b442-22a5-4329-9a1f-872e81ff6505/volumes/kubernetes.io~projected/kube-api-access-qb6tz major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b15479c-ad1d-4bae-af69-9409ce12337e/volumes/kubernetes.io~projected/kube-api-access-znp8b:{mountpoint:/var/lib/kubelet/pods/9b15479c-ad1d-4bae-af69-9409ce12337e/volumes/kubernetes.io~projected/kube-api-access-znp8b major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volumes/kubernetes.io~projected/kube-api-access-4n299:{mountpoint:/var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volumes/kubernetes.io~projected/kube-api-access-4n299 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/adedcfe7-c4f4-417e-ba6d-af46396d04f8/volumes/kubernetes.io~projected/kube-api-access-fpf64:{mountpoint:/var/lib/kubelet/pods/adedcfe7-c4f4-417e-ba6d-af46396d04f8/volumes/kubernetes.io~projected/kube-api-access-fpf64 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6ce3135-50d9-4dd2-b8e5-44af7397ec68/volumes/kubernetes.io~projected/kube-api-access-bjnhw:{mountpoint:/var/lib/kubelet/pods/b6ce3135-50d9-4dd2-b8e5-44af7397ec68/volumes/kubernetes.io~projected/kube-api-access-bjnhw major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cafe16db-7076-474d-826c-f908e4f4a71f/volumes/kubernetes.io~projected/kube-api-access-xq7bf:{mountpoint:/var/lib/kubelet/pods/cafe16db-7076-474d-826c-f908e4f4a71f/volumes/kubernetes.io~projected/kube-api-access-xq7bf major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cafe16db-7076-474d-826c-f908e4f4a71f/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/cafe16db-7076-474d-826c-f908e4f4a71f/volumes/kubernetes.io~secret/signing-key major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4baf7ce-ca52-4d1b-a721-39ba6be4647f/volumes/kubernetes.io~projected/kube-api-access-5bfvr:{mountpoint:/var/lib/kubelet/pods/d4baf7ce-ca52-4d1b-a721-39ba6be4647f/volumes/kubernetes.io~projected/kube-api-access-5bfvr major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dbdb63cc-bc6e-424c-a528-5232fdfc561c/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/dbdb63cc-bc6e-424c-a528-5232fdfc561c/volumes/kubernetes.io~secret/agent-certs major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4c82904-537f-49f8-bb3c-78fa7b00ccd6/volumes/kubernetes.io~projected/kube-api-access-kvmbp:{mountpoint:/var/lib/kubelet/pods/f4c82904-537f-49f8-bb3c-78fa7b00ccd6/volumes/kubernetes.io~projected/kube-api-access-kvmbp major:0 minor:216 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/ed41b8f6c5b5f6449f04d9349b995183556393e752548a4dc3c4d6a8c40cb22e/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/f79b0f7140dab09c712eef131edc2da146245df42e7201218d8060f22a731af0/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-105:{mountpoint:/var/lib/containers/storage/overlay/33c88f57da22104c4664097ebaaff7c51e40655209115ba0e38855f1d895fbda/merged major:0 minor:105 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/24f0f6fb81550fe513a3b7d2370d846a1483e2e963c528c40f22cfeb9f76a53a/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/fcd951822fa01ad8b8fa0bf0b1b42c1dcc8ce712a448608f744d6fe0f6354a56/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/5b31ef8fee62c9926eea6249da2c8ae56f6b39b005551fcf9e81affb7a33fc42/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/f05ee0da064ca8dfad088af22b567bfe69b0e41e97c7227e8960efd72a36d8cb/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/19d9b3c04cf2a358e16be5cdafd00bc36140f616be8636ff596865a0c69b8280/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/6f6ebc092cc3e11bde6fa7ab3a4e820f2b58db85227336e50ec476893da7e05e/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/67f4cf08b4b6ffc32ce1b651367580dec738d9702045d9e87d149916250f4fd3/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-138:{mountpoint:/var/lib/containers/storage/overlay/fecbbad65a7dda2a0d3bb1861345d077312b82ff283ee1a078b6686fa1b8611b/merged major:0 minor:138 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/edc472af3e2fb9115d928a191cbb43e35a49c0f128a0dd1c806132d76b73df30/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/d63603a8bfacb21a4d4d3ac75346f97034d6cf146f1943de55bf38c497bf4255/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/f4259dbfb71329dc7e5b8b7a55a58ca5938326bf483c1e6adfdcb1e14d0712e7/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/b194c3ec944d3cd81e3cb89e7820bde3c2bc2980b9ae3303fa0a36b8ef4aa249/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/6cbcdaf7cf792c918f0ced86a39f4ec3cbea1d11d1c4a28b44a45f288a01c75f/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/1442bb0df5d75383e7b0ef61ab78970d07c77230fe2e619e844497bd6eb48527/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-173:{mountpoint:/var/lib/containers/storage/overlay/f623d4aac3d2b6df37a490d59e153e4d63f2fe0083bad807f4bcf96284e2b1c7/merged major:0 minor:173 fsType:overlay blockSize:0} overlay_0-179:{mountpoint:/var/lib/containers/storage/overlay/13bd99a695b371e97c573836512ca22a90641e9a99f808ed8a0f37cb2de8cbdd/merged major:0 minor:179 fsType:overlay blockSize:0} overlay_0-184:{mountpoint:/var/lib/containers/storage/overlay/9e69203846884c96011898fb68a3017bef3d8cdd7db52604fdf1455a067d4c68/merged major:0 minor:184 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/0d8e5daa63a6088c579a788b4ce6bc8366e840c59caf55239cc1b5f6ff240aab/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/83e8b0263e5bb266d7ab0bc32b032c8ec912acca108b62377ed90c5ab99f2a3c/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/a5a0a8c4c96e042ff6ff5cebcc7afa428de6579f3ec9dd718c70e4e15dc2404f/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/44a3a044ab7d77cec0b1d31b49fe66d9e96b76bdf8b63b49283a28383e4cdb2f/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-220:{mountpoint:/var/lib/containers/storage/overlay/45523ba48478889308090ecc37c7156ac7a82b29b2bea62bac90f59bda29d1de/merged major:0 minor:220 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/b54611547d6dfe099ddfb98f2971605b8749e702ea5e638734c289d8e0cbc04c/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/1de6204ae39935aeb451dc64fe89a2a1e26bd8af88e93c1818c7b449bb52545f/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/84ba42a9f7374d72cfaa2645b081375183dafd2b5f94c675d1ed95e2bc3053e0/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/c70c7f09e90820aebcbda71c4baa39faa2175e2cda161443befcb2bff21744fe/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/397517f0306225678c21baf48bcf7e6e416a2e42cf05b200fa99916aca462436/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/ea90a1358fec715529ca84d8f18bdf301ff58c22b42b0f348c6649e3a9bfd247/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/dc00fdd5619173cd3257427d089d77d9194941aa941f484219b07f2b6fc94b46/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/22b9d9c36716bdfd1ae7012cd7e27910ff83b331298890e8f0f29192e5e77de7/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/e00efbbdc99270d0e51e8701f20544bbe6448fda1cc5e0e4f3aa8f465bc5b952/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/0d080a13660ba2d0491fc29b9e2addd14b643d848d0a048d48c97584779d314c/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/16118c0dcfd4555a701a4676bc9fd424c3b07aef81cff3b16efe66568e5fcea5/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/14dea30c2cb9e1fba102c4905a5bdfbb8042ba7035cebec1141cb77155ad320d/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/600d54e8613f2c0cfefbca1e3810bc57e00885da3ec1b08c70fb1186b6833bfd/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/006b5a73d7943774b1c4a26f208709970bb9a9775930343a92baeeddc284af4b/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/c67fc04edff5342cb575484deb39f2a479ca94fc6a89eeea5c5ff8002ed4f3d4/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/61d7e9bd047227335d0aa6096bc62a5880173761e6cb27b799096e297f80836b/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/f17a4c7fc2e1f5623e42dbfbd092db7adbbfa55bc7cf63c267ff1ed33e9c6027/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/4f0d2c92dd917da5f2cfb6701725173fd55ea0035a85fda3da19cacc0116fc99/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572034    4532 manager.go:217] Machine: {Timestamp:2026-10-31 07:29:09.571590998 +0000 UTC m=+0.073744780 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112410 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f7598153718f196a663ecb5f397d6 SystemUUID:ec2f7598-1537-18f1-96a6-63ecb5f397d6 BootID:13578c0c-7132-4df0-a6ad-18c68f4a8523 Filesystems:[{Device:/var/lib/kubelet/pods/cafe16db-7076-474d-826c-f908e4f4a71f/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:233 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/09202313-5fd7-46ab-b667-18188c9b28e7/volumes/kubernetes.io~projected/kube-api-access-nprzm DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b6ce3135-50d9-4dd2-b8e5-44af7397ec68/volumes/kubernetes.io~projected/kube-api-access-bjnhw DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/395e5c0f873407f0f6afad75c577b5435b5d9c19f9be4048b567325b58fb447a/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0fa0de5fc59ed79c13caa15a341e48b93450e339420b8151da16d13f476bee10/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/368da24d56f70bc8f4fe3fc6174f63d16567232e3a78b7dcee350002645fafbc/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4baf7ce-ca52-4d1b-a721-39ba6be4647f/volumes/kubernetes.io~projected/kube-api-access-5bfvr DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fbeb4283fa1a0d5e72b7bc850c83e42222985c7bc1e811ec4e6e836583a8ac65/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cafe16db-7076-474d-826c-f908e4f4a71f/volumes/kubernetes.io~projected/kube-api-access-xq7bf DeviceMajor:0 DeviceMinor:234 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/9b15479c-ad1d-4bae-af69-9409ce12337e/volumes/kubernetes.io~projected/kube-api-access-znp8b DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-173 DeviceMajor:0 DeviceMinor:173 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-138 DeviceMajor:0 DeviceMinor:138 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-184 DeviceMajor:0 DeviceMinor:184 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d4da9cf5d72bafa8b1a03c925db6c9ca1f7d225a12a0f76d714d6fce90f73a68/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/46ca8ac6-dfed-461e-90ed-17556a2e5272/volumes/kubernetes.io~projected/kube-api-access-99wxd DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/dbdb63cc-bc6e-424c-a528-5232fdfc561c/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c19671c9797d3022d0ec1596954989a36cd362ea11f6d2231ad3e94b7608aefa/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/65d23093-9f6e-4857-90be-3504d74e6dde/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f4c82904-537f-49f8-bb3c-78fa7b00ccd6/volumes/kubernetes.io~projected/kube-api-access-kvmbp DeviceMajor:0 DeviceMinor:216 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51974c85-319d-4074-b109-3927c8b0e147/volumes/kubernetes.io~projected/kube-api-access-7jndb DeviceMajor:0 DeviceMinor:228 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0910090c884c482f05e65f42c0d4d6e636bc5f27fb29799726adc95883b991a9/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~projected/kube-api-access-nghgg DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f3b4918a8db6722bff832095e27346be39aa09fc3f6e3ad57d40d5a9c198c614/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-179 DeviceMajor:0 DeviceMinor:179 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/583c98cf4ca216c2bcb77d9e45d3d6cbeacbf83325e0af838793af9a4fe02375/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/98e9b442-22a5-4329-9a1f-872e81ff6505/volumes/kubernetes.io~projected/kube-api-access-qb6tz DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aa0b9899-ff27-48dc-85a3-e88c23f8e897/volumes/kubernetes.io~projected/kube-api-access-4n299 DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/7b3156c0-bb65-443c-bfde-3ed0151d47ce/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/adedcfe7-c4f4-417e-ba6d-af46396d04f8/volumes/kubernetes.io~projected/kube-api-access-fpf64 DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/30297c93-42e8-45c4-91b4-bc87be152069/volumes/kubernetes.io~projected/kube-api-access-7qxq2 DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-105 DeviceMajor:0 DeviceMinor:105 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/831a61ee-1d95-484a-a379-434ea9284887/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/241ac3ab-0782-4990-87b8-d0199fc34945/volumes/kubernetes.io~projected/kube-api-access-twfwt DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/831a61ee-1d95-484a-a379-434ea9284887/volumes/kubernetes.io~projected/kube-api-access-rdvq9 DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fdbf44e3a2c45a72857ddb26eb6a8c0ca20c0a73c8b05a6df1a645177289a241/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-220 DeviceMajor:0 DeviceMinor:220 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8dbd7dfc6c19def94c43ca469fec4670c8c38ffa3454b40a67cc1cc84f07271f/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/23a52a8681039cf98cc7053c4639885ecbdfbb6a0cfd9c34446efc7db25d74db/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c/volumes/kubernetes.io~projected/kube-api-access-ktj2q DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cf1faebd06b43da90ccb06bfd0490bf51eca7690822fea73336a7e1173c32523/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0910090c884c482 MacAddress:12:0b:b9:11:dd:f3 Speed:10000 Mtu:8901} {Name:368da24d56f70bc MacAddress:ae:d2:2e:57:e3:a2 Speed:10000 Mtu:8901} {Name:8f75385dfa45c39 MacAddress:fe:de:2f:93:d7:25 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:28:4b:8c:45:df Speed:0 Mtu:9001} {Name:br-int MacAddress:c6:13:91:74:f2:ee Speed:0 Mtu:8901} {Name:d4da9cf5d72bafa MacAddress:9a:33:d7:21:31:53 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:28:4b:8c:45:df Speed:0 Mtu:9001} {Name:fbeb4283fa1a0d5 MacAddress:de:c1:38:1c:84:23 Speed:10000 Mtu:8901} {Name:fdbf44e3a2c45a7 MacAddress:22:8f:2d:a0:1c:44 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:5e:0c:12:cd:75:9e Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:8e:fb:70:59:9a:43 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572347    4532 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572414    4532 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572715    4532 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572736    4532 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-50.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572854    4532 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572861    4532 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572875    4532 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572898    4532 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.572977    4532 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.573142    4532 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.573266    4532 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.573303    4532 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.573315    4532 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.573326    4532 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.573334    4532 kubelet.go:405] "Adding apiserver pod source"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.573342    4532 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.574325    4532 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.574339    4532 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.575192    4532 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578431    4532 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578442    4532 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 07:29:09.578585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578573    4532 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578773    4532 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578801    4532 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578901    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578913    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578922    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578928    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578935    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578943    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578953    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578961    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578968    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578974    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.578993    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.579002    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.579027    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 07:29:09.579435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.579035    4532 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 07:29:09.582295 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.582281    4532 server.go:1263] "Started kubelet"
Oct 31 07:29:09.582415 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.582379    4532 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 07:29:09.582816 ip-10-0-133-50 systemd[1]: Started Kubernetes Kubelet.
Oct 31 07:29:09.584780 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.583854    4532 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 07:29:09.584780 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.583851    4532 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 07:29:09.584780 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.583991    4532 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 07:29:09.584780 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.584312    4532 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.586127    4532 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.586146    4532 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.586187    4532 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.586203    4532 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.586182    4532 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:23:30 +0000 UTC" deadline="2028-07-31 03:05:20.08553883 +0000 UTC"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.586220    4532 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15331h36m10.49932192s"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.586237    4532 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.586368    4532 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.587995    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" volumeName="kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588039    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="831a61ee-1d95-484a-a379-434ea9284887" volumeName="kubernetes.io/projected/831a61ee-1d95-484a-a379-434ea9284887-kube-api-access-rdvq9" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588056    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa0b9899-ff27-48dc-85a3-e88c23f8e897" volumeName="kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-env-overrides" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588068    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ce3135-50d9-4dd2-b8e5-44af7397ec68" volumeName="kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cni-binary-copy" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588108    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbdb63cc-bc6e-424c-a528-5232fdfc561c" volumeName="kubernetes.io/configmap/dbdb63cc-bc6e-424c-a528-5232fdfc561c-konnectivity-ca" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588121    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4c82904-537f-49f8-bb3c-78fa7b00ccd6" volumeName="kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588140    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="241ac3ab-0782-4990-87b8-d0199fc34945" volumeName="kubernetes.io/projected/241ac3ab-0782-4990-87b8-d0199fc34945-kube-api-access-twfwt" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588153    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30297c93-42e8-45c4-91b4-bc87be152069" volumeName="kubernetes.io/projected/30297c93-42e8-45c4-91b4-bc87be152069-kube-api-access-7qxq2" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588168    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" volumeName="kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588180    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="98e9b442-22a5-4329-9a1f-872e81ff6505" volumeName="kubernetes.io/projected/98e9b442-22a5-4329-9a1f-872e81ff6505-kube-api-access-qb6tz" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588192    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa0b9899-ff27-48dc-85a3-e88c23f8e897" volumeName="kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588202    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="adedcfe7-c4f4-417e-ba6d-af46396d04f8" volumeName="kubernetes.io/empty-dir/adedcfe7-c4f4-417e-ba6d-af46396d04f8-tmp-dir" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588211    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ce3135-50d9-4dd2-b8e5-44af7397ec68" volumeName="kubernetes.io/projected/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-kube-api-access-bjnhw" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588218    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dbdb63cc-bc6e-424c-a528-5232fdfc561c" volumeName="kubernetes.io/secret/dbdb63cc-bc6e-424c-a528-5232fdfc561c-agent-certs" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588227    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="241ac3ab-0782-4990-87b8-d0199fc34945" volumeName="kubernetes.io/configmap/241ac3ab-0782-4990-87b8-d0199fc34945-config-volume" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588234    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46ca8ac6-dfed-461e-90ed-17556a2e5272" volumeName="kubernetes.io/configmap/46ca8ac6-dfed-461e-90ed-17556a2e5272-iptables-alerter-script" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588241    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3156c0-bb65-443c-bfde-3ed0151d47ce" volumeName="kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-tmp" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588248    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3156c0-bb65-443c-bfde-3ed0151d47ce" volumeName="kubernetes.io/projected/7b3156c0-bb65-443c-bfde-3ed0151d47ce-kube-api-access-nghgg" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588255    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="831a61ee-1d95-484a-a379-434ea9284887" volumeName="kubernetes.io/secret/831a61ee-1d95-484a-a379-434ea9284887-cert" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588261    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa0b9899-ff27-48dc-85a3-e88c23f8e897" volumeName="kubernetes.io/secret/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588268    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ce3135-50d9-4dd2-b8e5-44af7397ec68" volumeName="kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-daemon-config" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588275    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4c82904-537f-49f8-bb3c-78fa7b00ccd6" volumeName="kubernetes.io/projected/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-api-access-kvmbp" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588284    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="241ac3ab-0782-4990-87b8-d0199fc34945" volumeName="kubernetes.io/empty-dir/241ac3ab-0782-4990-87b8-d0199fc34945-tmp-dir" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588294    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51974c85-319d-4074-b109-3927c8b0e147" volumeName="kubernetes.io/projected/51974c85-319d-4074-b109-3927c8b0e147-kube-api-access-7jndb" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588305    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cafe16db-7076-474d-826c-f908e4f4a71f" volumeName="kubernetes.io/projected/cafe16db-7076-474d-826c-f908e4f4a71f-kube-api-access-xq7bf" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588315    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4c82904-537f-49f8-bb3c-78fa7b00ccd6" volumeName="kubernetes.io/empty-dir/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-data-volume" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588322    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa0b9899-ff27-48dc-85a3-e88c23f8e897" volumeName="kubernetes.io/projected/aa0b9899-ff27-48dc-85a3-e88c23f8e897-kube-api-access-4n299" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588330    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09202313-5fd7-46ab-b667-18188c9b28e7" volumeName="kubernetes.io/projected/09202313-5fd7-46ab-b667-18188c9b28e7-kube-api-access-nprzm" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588339    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30297c93-42e8-45c4-91b4-bc87be152069" volumeName="kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588350    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" volumeName="kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588359    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" volumeName="kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588371    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65d23093-9f6e-4857-90be-3504d74e6dde" volumeName="kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588380    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa0b9899-ff27-48dc-85a3-e88c23f8e897" volumeName="kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-config" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588388    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cafe16db-7076-474d-826c-f908e4f4a71f" volumeName="kubernetes.io/configmap/cafe16db-7076-474d-826c-f908e4f4a71f-signing-cabundle" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588399    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30297c93-42e8-45c4-91b4-bc87be152069" volumeName="kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588407    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30297c93-42e8-45c4-91b4-bc87be152069" volumeName="kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-binary-copy" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588414    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46ca8ac6-dfed-461e-90ed-17556a2e5272" volumeName="kubernetes.io/projected/46ca8ac6-dfed-461e-90ed-17556a2e5272-kube-api-access-99wxd" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588421    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="98e9b442-22a5-4329-9a1f-872e81ff6505" volumeName="kubernetes.io/configmap/98e9b442-22a5-4329-9a1f-872e81ff6505-serviceca" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588427    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b15479c-ad1d-4bae-af69-9409ce12337e" volumeName="kubernetes.io/projected/9b15479c-ad1d-4bae-af69-9409ce12337e-kube-api-access-znp8b" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588435    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="adedcfe7-c4f4-417e-ba6d-af46396d04f8" volumeName="kubernetes.io/projected/adedcfe7-c4f4-417e-ba6d-af46396d04f8-kube-api-access-fpf64" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588441    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cafe16db-7076-474d-826c-f908e4f4a71f" volumeName="kubernetes.io/secret/cafe16db-7076-474d-826c-f908e4f4a71f-signing-key" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588447    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4baf7ce-ca52-4d1b-a721-39ba6be4647f" volumeName="kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588454    4532 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7b3156c0-bb65-443c-bfde-3ed0151d47ce" volumeName="kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-tuned" seLinuxMountContext=""
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588492    4532 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 07:29:09.589505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.588500    4532 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 07:29:09.591618 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.591261    4532 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 07:29:09.591618 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.591276    4532 factory.go:55] Registering systemd factory
Oct 31 07:29:09.591618 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.591287    4532 factory.go:223] Registration of the systemd container factory successfully
Oct 31 07:29:09.592619 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.592600    4532 factory.go:153] Registering CRI-O factory
Oct 31 07:29:09.592688 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.592622    4532 factory.go:223] Registration of the crio container factory successfully
Oct 31 07:29:09.592688 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.592645    4532 factory.go:103] Registering Raw factory
Oct 31 07:29:09.592688 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.592663    4532 manager.go:1199] Started watching for new ooms in manager
Oct 31 07:29:09.593619 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.593155    4532 manager.go:319] Starting recovery of all containers
Oct 31 07:29:09.616271 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.616217    4532 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 07:29:09.618488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.618444    4532 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 07:29:09.618488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.618491    4532 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 07:29:09.618648 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.618509    4532 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 07:29:09.618648 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.618546    4532 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 07:29:09.646106 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.646074    4532 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="56cf2da2b619257d2aaaa7a41c1c90c062e078ae1bcd0e9952641f41a95d50ed" exitCode=0
Oct 31 07:29:09.646106 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.646105    4532 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="b3799c44ed5b216d01ec6c19465a11617d404dd634b544a09859625081eb85f2" exitCode=0
Oct 31 07:29:09.646230 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.646115    4532 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="1f4d6dc3b57cd395c87d329e7752110a512e45987f769597deb04a4aa5aab464" exitCode=0
Oct 31 07:29:09.646230 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.646124    4532 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="7ea6fc44a5f9c4c098690cc4acf3ac9e49bf61ecdb3e13c4e26546d304bca88d" exitCode=0
Oct 31 07:29:09.646230 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.646133    4532 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="5c7ff9a477950e1cbe80f87f8e48320b25d9f0d70d57884c27d5e34364fc3bb1" exitCode=0
Oct 31 07:29:09.646230 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.646140    4532 generic.go:356] "Generic (PLEG): container finished" podID="30297c93-42e8-45c4-91b4-bc87be152069" containerID="9a0d8a1a1ff9cb28cbec29175a713ce64c75f512d7c49471afb6315596bd1a78" exitCode=0
Oct 31 07:29:09.652055 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.652032    4532 generic.go:356] "Generic (PLEG): container finished" podID="c09f5b3d9a6222e71966c227a0ba0115" containerID="d081c3b74415386d003e3a54049d486358a1fa6b169497ee7114ff3afe4d12f5" exitCode=0
Oct 31 07:29:09.672915 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.672898    4532 manager.go:324] Recovery completed
Oct 31 07:29:09.686441 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.686415    4532 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:29:09.687084 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.687070    4532 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:09.688733 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.688687    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:09.688733 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.688718    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:09.688733 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.688732    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:29:09.690162 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690144    4532 cpu_manager.go:226] "Starting" policy="none"
Oct 31 07:29:09.690162 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690160    4532 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 07:29:09.690292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690179    4532 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 07:29:09.690292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690154    4532 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:09.690380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690311    4532 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 07:29:09.690380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690324    4532 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 07:29:09.690380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690346    4532 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 07:29:09.690380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690354    4532 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 07:29:09.690380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690361    4532 policy_none.go:50] "Start"
Oct 31 07:29:09.690380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690372    4532 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 07:29:09.690380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690383    4532 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 07:29:09.690625 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690577    4532 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 07:29:09.690625 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690587    4532 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 07:29:09.690625 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.690595    4532 policy_none.go:44] "Start"
Oct 31 07:29:09.691222 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.691197    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:09.691298 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.691232    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:09.691298 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.691247    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:29:09.695357 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.695332    4532 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 07:29:09.698203 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698186    4532 manager.go:341] "Starting Device Plugin manager"
Oct 31 07:29:09.698277 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.698218    4532 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 07:29:09.698277 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698230    4532 server.go:87] "Starting device plugin registration server"
Oct 31 07:29:09.698572 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698557    4532 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 07:29:09.698632 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698573    4532 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 07:29:09.698721 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698706    4532 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 07:29:09.698875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698844    4532 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 07:29:09.698941 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698886    4532 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 07:29:09.698941 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698893    4532 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 07:29:09.699010 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.698946    4532 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T07:29:09.698872546Z","UUID":"af5170c4-9e0e-42bf-acc5-e0ccca54b07f","Handler":null,"Name":"","Endpoint":""}
Oct 31 07:29:09.700294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.700276    4532 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 07:29:09.700382 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.700302    4532 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 07:29:09.700420 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.700396    4532 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 07:29:09.700478 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.700432    4532 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-50.us-west-1.compute.internal\" not found"
Oct 31 07:29:09.719524 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.719007    4532 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"]
Oct 31 07:29:09.719614 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.719600    4532 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:09.720566 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.720551    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:09.720637 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.720577    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:09.720637 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.720593    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:29:09.720821 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.720811    4532 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:09.720973 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.720962    4532 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:09.721759 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.721743    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:09.721812 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.721771    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:09.721812 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.721782    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:29:09.721898 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.721743    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:09.721898 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.721854    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:09.721898 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.721875    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:29:09.721981 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.721918    4532 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.722118 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.722093    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"c09f5b3d9a6222e71966c227a0ba0115","Type":"ContainerStarted","Data":"ba5be1142f776978161c22bc4ed843c949d29b8bbe455ec5692568057efe856b"}
Oct 31 07:29:09.722156 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.722125    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"c09f5b3d9a6222e71966c227a0ba0115","Type":"ContainerDied","Data":"d081c3b74415386d003e3a54049d486358a1fa6b169497ee7114ff3afe4d12f5"}
Oct 31 07:29:09.722156 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.722138    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"c09f5b3d9a6222e71966c227a0ba0115","Type":"ContainerStarted","Data":"28a4b811811ea5372832d5c95c1a81701bdd4046b0a934ef316f516a751e4d58"}
Oct 31 07:29:09.722156 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.722151    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"0fc2efca0f9b45f2393e3897a932b4b4","Type":"ContainerStarted","Data":"d8b568abb584898e2462161b12a525ae7b2e274723ba0577835307c44d12b899"}
Oct 31 07:29:09.722240 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.722164    4532 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:09.722240 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.722162    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal" event={"ID":"0fc2efca0f9b45f2393e3897a932b4b4","Type":"ContainerStarted","Data":"5d823db85375456e04c276e46a2e2ff9c9d72b983379dcd81113f9b313303a01"}
Oct 31 07:29:09.723077 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.723060    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:09.723144 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.723088    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:09.723144 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.723098    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:29:09.723251 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.723210    4532 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-50.us-west-1.compute.internal\" not found" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.783532 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.783511    4532 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 07:29:09.786551 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.786527    4532 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:09.786551 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.786534    4532 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:09.799001 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.798981    4532 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 07:29:09.800876 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.800860    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 31 07:29:09.800936 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.800887    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 07:29:09.800936 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.800898    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 31 07:29:09.800936 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.800927    4532 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.828738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.828718    4532 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:09.836185 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.836164    4532 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 07:29:09.884306 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.884284    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0fc2efca0f9b45f2393e3897a932b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"0fc2efca0f9b45f2393e3897a932b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.884414 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.884318    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.884414 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.884345    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.887426 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.887406    4532 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.929364 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.929344    4532 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.929427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.929397    4532 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.929893 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.929879    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeNotReady"
Oct 31 07:29:09.929947 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.929907    4532 setters.go:546] "Node became not ready" node="ip-10-0-133-50.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-31T07:29:09Z","lastTransitionTime":"2026-10-31T07:29:09Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 31 07:29:09.944629 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.944581    4532 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:09.944719 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:09.944625    4532 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.944719 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.944641    4532 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.945624 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.945607    4532 nodeinfomanager.go:471] existing CSINode "ip-10-0-133-50.us-west-1.compute.internal" is owned by different node (oldNodeID="75b38aa3-4174-4310-9d02-f20259a0eb09", newNodeID=""), cleaning up...
Oct 31 07:29:09.984950 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.984929    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0fc2efca0f9b45f2393e3897a932b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"0fc2efca0f9b45f2393e3897a932b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.985034 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.984958    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.985034 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.984984    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.985117 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.985041    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.985117 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.985047    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0fc2efca0f9b45f2393e3897a932b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"0fc2efca0f9b45f2393e3897a932b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:09.985175 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:09.985109    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c09f5b3d9a6222e71966c227a0ba0115-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" (UID: \"c09f5b3d9a6222e71966c227a0ba0115\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:10.003644 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.003623    4532 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:10.003708 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.003655    4532 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:10.574043 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.574009    4532 apiserver.go:52] "Watching apiserver"
Oct 31 07:29:10.662704 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.662677    4532 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:10.662999 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.662767    4532 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:10.677643 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.677624    4532 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 07:29:10.678075 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.678058    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-x4b5m-ps6h6","openshift-multus/multus-b76wl","openshift-multus/multus-additional-cni-plugins-c6r54","openshift-image-registry/node-ca-w5t6j","openshift-service-ca/service-ca-5fffc97bdf-xrxjs","openshift-multus/network-metrics-daemon-w7gt6","openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k","openshift-ovn-kubernetes/ovnkube-node-94t84","openshift-cluster-node-tuning-operator/tuned-7mhkh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal","openshift-dns/dns-default-mqsmg","openshift-dns/node-resolver-4k94x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh","kube-system/konnectivity-agent-zgfmt","openshift-ingress-canary/ingress-canary-qdpz4","openshift-network-diagnostics/network-check-target-7cl42","kube-system/global-pull-secret-syncer-6qkpn","openshift-network-operator/iptables-alerter-jk2nc","openshift-insights/insights-runtime-extractor-5gj7z"]
Oct 31 07:29:10.682647 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.682617    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:10.685149 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.685131    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.685844 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.685827    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.688303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688284    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cni-binary-copy\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.688404 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688310    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-kubernetes\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.688404 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688325    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-conf\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.688404 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688340    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-node-log\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.688404 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688355    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-netd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.688626 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688425    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvmbp\" (UniqueName: \"kubernetes.io/projected/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-api-access-kvmbp\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.688626 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688445    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.688626 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688527    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qxq2\" (UniqueName: \"kubernetes.io/projected/30297c93-42e8-45c4-91b4-bc87be152069-kube-api-access-7qxq2\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.688626 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688561    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ca8ac6-dfed-461e-90ed-17556a2e5272-host-slash\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:29:10.688626 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688588    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-system-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688633    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjnhw\" (UniqueName: \"kubernetes.io/projected/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-kube-api-access-bjnhw\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688670    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-env-overrides\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688718    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-var-lib-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688735    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-kubelet-config\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688751    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-kubelet\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688765    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-log-socket\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688781    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znp8b\" (UniqueName: \"kubernetes.io/projected/9b15479c-ad1d-4bae-af69-9409ce12337e-kube-api-access-znp8b\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688816    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831a61ee-1d95-484a-a379-434ea9284887-cert\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:29:10.688879 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688866    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e9b442-22a5-4329-9a1f-872e81ff6505-host\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688921    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-etc-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688958    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.688980    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/241ac3ab-0782-4990-87b8-d0199fc34945-config-volume\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689011    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/241ac3ab-0782-4990-87b8-d0199fc34945-tmp-dir\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689054    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb6tz\" (UniqueName: \"kubernetes.io/projected/98e9b442-22a5-4329-9a1f-872e81ff6505-kube-api-access-qb6tz\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689077    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-run\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689130    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/241ac3ab-0782-4990-87b8-d0199fc34945-tmp-dir\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689208    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689229    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689244    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-modprobe-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.689267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689259    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689274    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-bin\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689310    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-device-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689365    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-conf-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689396    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-var-lib-kubelet\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689418    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktj2q\" (UniqueName: \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689437    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-cnibin\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689559    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689584    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-systemd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689608    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-registration-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689651    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-data-volume\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689667    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-bin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689692    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-kubelet\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689716    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-dbus\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.689747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689750    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-script-lib\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689755    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-data-volume\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689799    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-socket-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689818    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-sys-fs\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689834    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-crio-socket\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689862    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99wxd\" (UniqueName: \"kubernetes.io/projected/46ca8ac6-dfed-461e-90ed-17556a2e5272-kube-api-access-99wxd\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.689891    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twfwt\" (UniqueName: \"kubernetes.io/projected/241ac3ab-0782-4990-87b8-d0199fc34945-kube-api-access-twfwt\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690000    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690021    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690035    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-slash\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690053    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adedcfe7-c4f4-417e-ba6d-af46396d04f8-tmp-dir\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690121    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adedcfe7-c4f4-417e-ba6d-af46396d04f8-tmp-dir\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690126    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-netns\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690153    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-multus\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690179    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb63cc-bc6e-424c-a528-5232fdfc561c-agent-certs\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690200    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jndb\" (UniqueName: \"kubernetes.io/projected/51974c85-319d-4074-b109-3927c8b0e147-kube-api-access-7jndb\") pod \"migrator-66df5d496b-f9w6k\" (UID: \"51974c85-319d-4074-b109-3927c8b0e147\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690230    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-systemd\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690263    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-k8s-cni-cncf-io\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690286    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-multus-certs\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690305    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb63cc-bc6e-424c-a528-5232fdfc561c-konnectivity-ca\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690321    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysconfig\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.690352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690354    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-sys\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690395    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690418    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690473    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690506    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-etc-selinux\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690535    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-binary-copy\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690558    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690574    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-daemon-config\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690588    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-host\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690606    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-tuned\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690664    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/adedcfe7-c4f4-417e-ba6d-af46396d04f8-hosts-file\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690686    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e9b442-22a5-4329-9a1f-872e81ff6505-serviceca\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690705    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690726    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-netns\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690741    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-ovn\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690756    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-socket-dir-parent\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690781    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-hostroot\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690781    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-tuned\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690816    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-etc-kubernetes\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690843    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdvq9\" (UniqueName: \"kubernetes.io/projected/831a61ee-1d95-484a-a379-434ea9284887-kube-api-access-rdvq9\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690882    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-lib-modules\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690910    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690932    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.690978    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691011    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-tmp\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691053    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nghgg\" (UniqueName: \"kubernetes.io/projected/7b3156c0-bb65-443c-bfde-3ed0151d47ce-kube-api-access-nghgg\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691070    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-system-cni-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691099    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-config\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691105    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b3156c0-bb65-443c-bfde-3ed0151d47ce-tmp\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691225    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691256    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cnibin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691275    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-systemd-units\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.691303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691306    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n299\" (UniqueName: \"kubernetes.io/projected/aa0b9899-ff27-48dc-85a3-e88c23f8e897-kube-api-access-4n299\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.692728 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691334    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.692728 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691436    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.692728 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691487    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nprzm\" (UniqueName: \"kubernetes.io/projected/09202313-5fd7-46ab-b667-18188c9b28e7-kube-api-access-nprzm\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:10.692728 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691509    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovn-node-metrics-cert\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.692728 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691537    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ca8ac6-dfed-461e-90ed-17556a2e5272-iptables-alerter-script\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:29:10.692728 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.691555    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpf64\" (UniqueName: \"kubernetes.io/projected/adedcfe7-c4f4-417e-ba6d-af46396d04f8-kube-api-access-fpf64\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:29:10.725930 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.725371    4532 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:10.725930 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.725494    4532 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 07:29:10.725930 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.725526    4532 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:10.725930 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.725767    4532 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal"
Oct 31 07:29:10.730487 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.730075    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7fk4f\""
Oct 31 07:29:10.730809 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.730781    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.730893 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.730808    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 07:29:10.730893 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.730853    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.731274 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.731250    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 07:29:10.731426 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.731397    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-binary-copy\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.732236 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.732218    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-ldrdc\""
Oct 31 07:29:10.732302 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.732259    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pvmcp\""
Oct 31 07:29:10.732438 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.732417    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-798mv\""
Oct 31 07:29:10.734477 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.734451    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 07:29:10.734649 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.734635    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bpwql\""
Oct 31 07:29:10.734861 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.734850    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 07:29:10.734932 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.734917    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-46dh7\""
Oct 31 07:29:10.735046 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.735022    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-5qqnh\""
Oct 31 07:29:10.735625 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.735612    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 07:29:10.735687 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.735671    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gttml\""
Oct 31 07:29:10.735792 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.735778    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w8pvl\""
Oct 31 07:29:10.739124 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.739106    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cni-binary-copy\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.741228 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.741211    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/65d23093-9f6e-4857-90be-3504d74e6dde-original-pull-secret\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.741297 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.741239    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-daemon-config\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.742414 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.742392    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovn-node-metrics-cert\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.772233 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772122    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 07:29:10.772233 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772214    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6g5bx\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772704    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772755    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772802    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772846    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g75wk\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772936    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kwsqb\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.772985    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-crxcw\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.773000    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.773018    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xb4h7\""
Oct 31 07:29:10.773053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.773041    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jtbjn\""
Oct 31 07:29:10.773354 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.773068    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 07:29:10.773354 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.773081    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pkrx7\""
Oct 31 07:29:10.773354 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.773083    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-m5kd5\""
Oct 31 07:29:10.777207 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.777185    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twfwt\" (UniqueName: \"kubernetes.io/projected/241ac3ab-0782-4990-87b8-d0199fc34945-kube-api-access-twfwt\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.779661 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.779601    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/831a61ee-1d95-484a-a379-434ea9284887-cert\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:29:10.779767 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.779679    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-845kx\""
Oct 31 07:29:10.780878 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.780857    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dbdb63cc-bc6e-424c-a528-5232fdfc561c-agent-certs\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:29:10.786705 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.786686    4532 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 07:29:10.787004 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.786988    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpf64\" (UniqueName: \"kubernetes.io/projected/adedcfe7-c4f4-417e-ba6d-af46396d04f8-kube-api-access-fpf64\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:29:10.792196 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792175    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-netns\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792273 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792208    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-ovn\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792273 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792232    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-socket-dir-parent\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792273 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792255    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-hostroot\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792278    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-etc-kubernetes\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792279    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-netns\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792291    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-ovn\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792301    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-lib-modules\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792321    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-socket-dir-parent\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792340    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-hostroot\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792346    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792351    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-etc-kubernetes\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792374    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-system-cni-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.792428 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792401    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cnibin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792445    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-lib-modules\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792448    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-system-cni-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792459    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-cnibin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.792452    4532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792488    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-systemd-units\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792518    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.792548    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:11.292530001 +0000 UTC m=+1.794683793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792575    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-systemd-units\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.792585    4532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792577    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-kubernetes\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792612    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-conf\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792617    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-kubernetes\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.792633    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:11.292618509 +0000 UTC m=+1.794772292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792670    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-node-log\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792704    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-netd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792724    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-node-log\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792731    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ca8ac6-dfed-461e-90ed-17556a2e5272-host-slash\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792764    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-netd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792766    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-system-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792733    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-conf\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792765    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ca8ac6-dfed-461e-90ed-17556a2e5272-host-slash\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792798    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-var-lib-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792818    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-system-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792828    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-kubelet-config\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.792840 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792850    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-kubelet\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792861    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-var-lib-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792871    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-log-socket\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792894    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e9b442-22a5-4329-9a1f-872e81ff6505-host\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792901    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-kubelet\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792912    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-kubelet-config\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792916    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-etc-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792936    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e9b442-22a5-4329-9a1f-872e81ff6505-host\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792927    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-log-socket\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792949    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-etc-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792953    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-run\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792979    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.792991    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-run\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793002    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-modprobe-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793028    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq7bf\" (UniqueName: \"kubernetes.io/projected/cafe16db-7076-474d-826c-f908e4f4a71f-kube-api-access-xq7bf\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.793044    4532 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793090    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:10.793104    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs podName:09202313-5fd7-46ab-b667-18188c9b28e7 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:11.2930757 +0000 UTC m=+1.795229483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs") pod "network-metrics-daemon-w7gt6" (UID: "09202313-5fd7-46ab-b667-18188c9b28e7") : secret "metrics-daemon-secret" not found
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793052    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-run-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793108    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-modprobe-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793130    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-bin\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793155    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-device-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793178    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-conf-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793203    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-var-lib-kubelet\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793222    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-cni-bin\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793235    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-device-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793228    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-cnibin\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793265    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-conf-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793273    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793280    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-cnibin\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793293    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-var-lib-kubelet\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793298    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-systemd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793330    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-registration-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793355    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-bin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793367    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793331    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-systemd\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793378    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-kubelet\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793412    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-kubelet\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793413    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-dbus\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793441    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-socket-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793448    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-bin\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793454    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/65d23093-9f6e-4857-90be-3504d74e6dde-dbus\") pod \"global-pull-secret-syncer-6qkpn\" (UID: \"65d23093-9f6e-4857-90be-3504d74e6dde\") " pod="kube-system/global-pull-secret-syncer-6qkpn"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793417    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-registration-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793491    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-sys-fs\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793515    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-crio-socket\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793535    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-socket-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793540    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793571    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-slash\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793593    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-multus-cni-dir\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793597    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-sys-fs\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793599    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-netns\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793630    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-host-slash\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793669    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-netns\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793697    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-multus\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793726    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-crio-socket\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793729    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-systemd\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793761    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-systemd\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793766    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-k8s-cni-cncf-io\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793728    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-var-lib-cni-multus\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793791    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-multus-certs\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793815    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysconfig\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793836    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-sys\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793860    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cafe16db-7076-474d-826c-f908e4f4a71f-signing-cabundle\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793862    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-multus-certs\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793884    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysconfig\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793907    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-host-run-k8s-cni-cncf-io\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.793889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793927    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-sys\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793957    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.793996    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794020    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-etc-selinux\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794045    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794054    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794070    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-host\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794072    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa0b9899-ff27-48dc-85a3-e88c23f8e897-run-openvswitch\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794085    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9b15479c-ad1d-4bae-af69-9409ce12337e-etc-selinux\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794097    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/adedcfe7-c4f4-417e-ba6d-af46396d04f8-hosts-file\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794104    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30297c93-42e8-45c4-91b4-bc87be152069-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794122    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794123    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-host\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794152    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/adedcfe7-c4f4-417e-ba6d-af46396d04f8-hosts-file\") pod \"node-resolver-4k94x\" (UID: \"adedcfe7-c4f4-417e-ba6d-af46396d04f8\") " pod="openshift-dns/node-resolver-4k94x"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794161    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cafe16db-7076-474d-826c-f908e4f4a71f-signing-key\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794224    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b3156c0-bb65-443c-bfde-3ed0151d47ce-etc-sysctl-d\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.795831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.794410    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cafe16db-7076-474d-826c-f908e4f4a71f-signing-key\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.831866    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.831873    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.831931    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.831941    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.831981    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832018    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832026    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832035    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832026    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832045    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832057    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832082    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832095    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 07:29:10.832145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832105    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832159    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832173    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832205    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832236    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832238    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832254    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832281    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832283    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832293    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832310    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832320    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832547    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832615    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 07:29:10.832756 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.832654    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.833293 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.833113    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.833351 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.833295    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.833351 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.833328    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.833554 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.833537    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 07:29:10.833616 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.833556    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99wxd\" (UniqueName: \"kubernetes.io/projected/46ca8ac6-dfed-461e-90ed-17556a2e5272-kube-api-access-99wxd\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:29:10.833616 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.833580    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 07:29:10.833773 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.833758    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 07:29:10.835011 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.834227    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cafe16db-7076-474d-826c-f908e4f4a71f-signing-cabundle\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:10.835011 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.834532    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nprzm\" (UniqueName: \"kubernetes.io/projected/09202313-5fd7-46ab-b667-18188c9b28e7-kube-api-access-nprzm\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:10.835011 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.834568    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvmbp\" (UniqueName: \"kubernetes.io/projected/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-api-access-kvmbp\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.835011 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.834662    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nghgg\" (UniqueName: \"kubernetes.io/projected/7b3156c0-bb65-443c-bfde-3ed0151d47ce-kube-api-access-nghgg\") pod \"tuned-7mhkh\" (UID: \"7b3156c0-bb65-443c-bfde-3ed0151d47ce\") " pod="openshift-cluster-node-tuning-operator/tuned-7mhkh"
Oct 31 07:29:10.835264 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.835217    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qxq2\" (UniqueName: \"kubernetes.io/projected/30297c93-42e8-45c4-91b4-bc87be152069-kube-api-access-7qxq2\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.837547 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.835501    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdvq9\" (UniqueName: \"kubernetes.io/projected/831a61ee-1d95-484a-a379-434ea9284887-kube-api-access-rdvq9\") pod \"ingress-canary-qdpz4\" (UID: \"831a61ee-1d95-484a-a379-434ea9284887\") " pod="openshift-ingress-canary/ingress-canary-qdpz4"
Oct 31 07:29:10.837547 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.835640    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjnhw\" (UniqueName: \"kubernetes.io/projected/b6ce3135-50d9-4dd2-b8e5-44af7397ec68-kube-api-access-bjnhw\") pod \"multus-b76wl\" (UID: \"b6ce3135-50d9-4dd2-b8e5-44af7397ec68\") " pod="openshift-multus/multus-b76wl"
Oct 31 07:29:10.837547 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.835699    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n299\" (UniqueName: \"kubernetes.io/projected/aa0b9899-ff27-48dc-85a3-e88c23f8e897-kube-api-access-4n299\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.837547 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.836523    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 07:29:10.838918 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.838899    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.838918 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.838915    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-env-overrides\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.839017 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.838970    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znp8b\" (UniqueName: \"kubernetes.io/projected/9b15479c-ad1d-4bae-af69-9409ce12337e-kube-api-access-znp8b\") pod \"aws-ebs-csi-driver-node-45gjh\" (UID: \"9b15479c-ad1d-4bae-af69-9409ce12337e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45gjh"
Oct 31 07:29:10.839254 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.839234    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bfvr\" (UniqueName: \"kubernetes.io/projected/d4baf7ce-ca52-4d1b-a721-39ba6be4647f-kube-api-access-5bfvr\") pod \"network-check-target-7cl42\" (UID: \"d4baf7ce-ca52-4d1b-a721-39ba6be4647f\") " pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:10.839407 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.839393    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.839450 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.839419    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/241ac3ab-0782-4990-87b8-d0199fc34945-config-volume\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:10.839450 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.839426    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.840006 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.839987    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jndb\" (UniqueName: \"kubernetes.io/projected/51974c85-319d-4074-b109-3927c8b0e147-kube-api-access-7jndb\") pod \"migrator-66df5d496b-f9w6k\" (UID: \"51974c85-319d-4074-b109-3927c8b0e147\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-f9w6k"
Oct 31 07:29:10.840075 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.840021    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-script-lib\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.840075 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.840032    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktj2q\" (UniqueName: \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\") pod \"periodic-gathering-x4b5m-ps6h6\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") " pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:29:10.840141 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.840126    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb6tz\" (UniqueName: \"kubernetes.io/projected/98e9b442-22a5-4329-9a1f-872e81ff6505-kube-api-access-qb6tz\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:29:10.841120 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.841101    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e9b442-22a5-4329-9a1f-872e81ff6505-serviceca\") pod \"node-ca-w5t6j\" (UID: \"98e9b442-22a5-4329-9a1f-872e81ff6505\") " pod="openshift-image-registry/node-ca-w5t6j"
Oct 31 07:29:10.841177 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.841128    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb63cc-bc6e-424c-a528-5232fdfc561c-konnectivity-ca\") pod \"konnectivity-agent-zgfmt\" (UID: \"dbdb63cc-bc6e-424c-a528-5232fdfc561c\") " pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:29:10.841253 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.841241    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa0b9899-ff27-48dc-85a3-e88c23f8e897-ovnkube-config\") pod \"ovnkube-node-94t84\" (UID: \"aa0b9899-ff27-48dc-85a3-e88c23f8e897\") " pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:10.841359 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.841348    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:10.842277 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.842265    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ca8ac6-dfed-461e-90ed-17556a2e5272-iptables-alerter-script\") pod \"iptables-alerter-jk2nc\" (UID: \"46ca8ac6-dfed-461e-90ed-17556a2e5272\") " pod="openshift-network-operator/iptables-alerter-jk2nc"
Oct 31 07:29:10.842311 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.842274    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30297c93-42e8-45c4-91b4-bc87be152069-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c6r54\" (UID: \"30297c93-42e8-45c4-91b4-bc87be152069\") " pod="openshift-multus/multus-additional-cni-plugins-c6r54"
Oct 31 07:29:10.850622 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:10.850600    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq7bf\" (UniqueName: \"kubernetes.io/projected/cafe16db-7076-474d-826c-f908e4f4a71f-kube-api-access-xq7bf\") pod \"service-ca-5fffc97bdf-xrxjs\" (UID: \"cafe16db-7076-474d-826c-f908e4f4a71f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-xrxjs"
Oct 31 07:29:11.209831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.209771    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4k94x_adedcfe7-c4f4-417e-ba6d-af46396d04f8/dns-node-resolver/0.log"
Oct 31 07:29:11.298567 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.298538    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:11.298683 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.298570    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:11.298683 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.298595    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:11.298683 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:11.298674    4532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:11.298828 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:11.298737    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:12.298724781 +0000 UTC m=+2.800878549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:11.298828 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:11.298676    4532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:11.298916 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:11.298833    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:12.298816944 +0000 UTC m=+2.800970715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:11.298949 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.298936    4532 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 07:29:11.300946 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.300921    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09202313-5fd7-46ab-b667-18188c9b28e7-metrics-certs\") pod \"network-metrics-daemon-w7gt6\" (UID: \"09202313-5fd7-46ab-b667-18188c9b28e7\") " pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:11.443307 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.443283    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:11.584517 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:11.584496    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7gt6"
Oct 31 07:29:12.020032 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:12.019961    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w7gt6"]
Oct 31 07:29:12.023361 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:12.023332    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09202313_5fd7_46ab_b667_18188c9b28e7.slice/crio-53773c360c4870b1640f023660f5519a852f09229ce567006e812c63f2e1d281 WatchSource:0}: Error finding container 53773c360c4870b1640f023660f5519a852f09229ce567006e812c63f2e1d281: Status 404 returned error can't find the container with id 53773c360c4870b1640f023660f5519a852f09229ce567006e812c63f2e1d281
Oct 31 07:29:12.025581 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:12.025563    4532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:29:12.306394 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:12.306353    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:12.306662 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:12.306636    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:12.306939 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:12.306923    4532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:12.307093 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:12.307083    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:14.307055916 +0000 UTC m=+4.809209686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:12.307596 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:12.307577    4532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:12.307741 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:12.307731    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:14.307715626 +0000 UTC m=+4.809869411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:12.607348 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:12.607300    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w5t6j_98e9b442-22a5-4329-9a1f-872e81ff6505/node-ca/0.log"
Oct 31 07:29:12.669384 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:12.669354    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w7gt6" event={"ID":"09202313-5fd7-46ab-b667-18188c9b28e7","Type":"ContainerStarted","Data":"53773c360c4870b1640f023660f5519a852f09229ce567006e812c63f2e1d281"}
Oct 31 07:29:13.206902 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:13.206873    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qdpz4_831a61ee-1d95-484a-a379-434ea9284887/serve-healthcheck-canary/0.log"
Oct 31 07:29:13.608622 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:13.608596    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-f9w6k_51974c85-319d-4074-b109-3927c8b0e147/migrator/0.log"
Oct 31 07:29:13.674101 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:13.674074    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w7gt6" event={"ID":"09202313-5fd7-46ab-b667-18188c9b28e7","Type":"ContainerStarted","Data":"df17c4bacb3e6b87dade1290546cd1cf66bb26283562d6f278f64f756e815f5e"}
Oct 31 07:29:13.674235 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:13.674120    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w7gt6" event={"ID":"09202313-5fd7-46ab-b667-18188c9b28e7","Type":"ContainerStarted","Data":"2bb4b2a2cd0b0e2e85770fe916fcd6047015797ce2bd98b7ca94a8a960e46528"}
Oct 31 07:29:14.318522 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:14.318492    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:14.318675 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:14.318528    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:14.318675 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:14.318636    4532 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:14.318746 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:14.318688    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls podName:f4c82904-537f-49f8-bb3c-78fa7b00ccd6 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:18.318676346 +0000 UTC m=+8.820830126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5gj7z" (UID: "f4c82904-537f-49f8-bb3c-78fa7b00ccd6") : secret "insights-runtime-extractor-tls" not found
Oct 31 07:29:14.318746 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:14.318636    4532 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 07:29:14.318813 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:14.318761    4532 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls podName:241ac3ab-0782-4990-87b8-d0199fc34945 nodeName:}" failed. No retries permitted until 2026-10-31 07:29:18.318747143 +0000 UTC m=+8.820900927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls") pod "dns-default-mqsmg" (UID: "241ac3ab-0782-4990-87b8-d0199fc34945") : secret "dns-default-metrics-tls" not found
Oct 31 07:29:18.338795 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.338759    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:18.339275 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.338802    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:18.341110 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.341082    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/241ac3ab-0782-4990-87b8-d0199fc34945-metrics-tls\") pod \"dns-default-mqsmg\" (UID: \"241ac3ab-0782-4990-87b8-d0199fc34945\") " pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:18.341217 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.341118    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f4c82904-537f-49f8-bb3c-78fa7b00ccd6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5gj7z\" (UID: \"f4c82904-537f-49f8-bb3c-78fa7b00ccd6\") " pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:18.429712 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.429686    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:18.485551 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.485518    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:18.486603 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.486584    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5gj7z"
Oct 31 07:29:18.624704 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.624638    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:18.687179 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.687157    4532 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 07:29:18.909914 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.909853    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5gj7z"]
Oct 31 07:29:18.913006 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:18.912982    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c82904_537f_49f8_bb3c_78fa7b00ccd6.slice/crio-ce9c628b3ffcb51fefe185ed0ba56393e333ed0aab8d906ca8062b904a337a9b WatchSource:0}: Error finding container ce9c628b3ffcb51fefe185ed0ba56393e333ed0aab8d906ca8062b904a337a9b: Status 404 returned error can't find the container with id ce9c628b3ffcb51fefe185ed0ba56393e333ed0aab8d906ca8062b904a337a9b
Oct 31 07:29:18.941659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:18.941579    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mqsmg"]
Oct 31 07:29:18.945512 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:18.945485    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241ac3ab_0782_4990_87b8_d0199fc34945.slice/crio-5a6b8256d9713c97a32e460c33b5ede3f34a288a3d52c3acfb9a113f71898172 WatchSource:0}: Error finding container 5a6b8256d9713c97a32e460c33b5ede3f34a288a3d52c3acfb9a113f71898172: Status 404 returned error can't find the container with id 5a6b8256d9713c97a32e460c33b5ede3f34a288a3d52c3acfb9a113f71898172
Oct 31 07:29:19.694082 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:19.694045    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gj7z" event={"ID":"f4c82904-537f-49f8-bb3c-78fa7b00ccd6","Type":"ContainerStarted","Data":"f9e437d4d24704f2ba9d0305ae86e3d7baf497b5ab5e08ff73258b73936386a8"}
Oct 31 07:29:19.694519 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:19.694134    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gj7z" event={"ID":"f4c82904-537f-49f8-bb3c-78fa7b00ccd6","Type":"ContainerStarted","Data":"ce9c628b3ffcb51fefe185ed0ba56393e333ed0aab8d906ca8062b904a337a9b"}
Oct 31 07:29:19.695178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:19.695153    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqsmg" event={"ID":"241ac3ab-0782-4990-87b8-d0199fc34945","Type":"ContainerStarted","Data":"5a6b8256d9713c97a32e460c33b5ede3f34a288a3d52c3acfb9a113f71898172"}
Oct 31 07:29:20.367696 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:20.367660    4532 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-133-50.us-west-1.compute.internal" event="NodeReady"
Oct 31 07:29:20.700223 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:20.700145    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gj7z" event={"ID":"f4c82904-537f-49f8-bb3c-78fa7b00ccd6","Type":"ContainerStarted","Data":"4422e1739b6f3890e1bcbf315cef71e842c580cd4e826776f674c5c19eb854ff"}
Oct 31 07:29:21.705798 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:21.705753    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqsmg" event={"ID":"241ac3ab-0782-4990-87b8-d0199fc34945","Type":"ContainerStarted","Data":"01073766ad406d9a0b145bf1aa73d5419e9fec8a2b1a69c67f8227730b4e0fd1"}
Oct 31 07:29:21.706218 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:21.705821    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqsmg" event={"ID":"241ac3ab-0782-4990-87b8-d0199fc34945","Type":"ContainerStarted","Data":"bd1f42d774bc9ad6b7112da62f868553747e15557d1803f1b2288633ad8fa442"}
Oct 31 07:29:21.706609 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:21.706586    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:22.133135 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.133108    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75fbcd66cd-wf4ks"]
Oct 31 07:29:22.134667 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.134651    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.165838 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.165809    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a431397d-d633-41a0-9c55-9291e03d4b4b-trusted-ca\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.165838 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.165838    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a431397d-d633-41a0-9c55-9291e03d4b4b-installation-pull-secrets\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.166036 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.165870    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a431397d-d633-41a0-9c55-9291e03d4b4b-ca-trust-extracted\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.166036 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.165946    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krwhl\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-kube-api-access-krwhl\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.166036 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.165995    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-bound-sa-token\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.166132 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.166042    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a431397d-d633-41a0-9c55-9291e03d4b4b-image-registry-private-configuration\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.166132 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.166100    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-registry-tls\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.166132 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.166120    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a431397d-d633-41a0-9c55-9291e03d4b4b-registry-certificates\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.186572 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.186553    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 07:29:22.200344 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.200323    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w5nxl\""
Oct 31 07:29:22.236072 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.236053    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 07:29:22.236154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.236098    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 07:29:22.267369 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267343    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a431397d-d633-41a0-9c55-9291e03d4b4b-image-registry-private-configuration\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267459 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267383    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-registry-tls\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267459 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267411    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a431397d-d633-41a0-9c55-9291e03d4b4b-registry-certificates\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267459 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267439    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a431397d-d633-41a0-9c55-9291e03d4b4b-trusted-ca\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267648 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267486    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a431397d-d633-41a0-9c55-9291e03d4b4b-installation-pull-secrets\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267648 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267522    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a431397d-d633-41a0-9c55-9291e03d4b4b-ca-trust-extracted\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267750 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267645    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krwhl\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-kube-api-access-krwhl\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267750 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267682    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-bound-sa-token\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.267995 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.267973    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a431397d-d633-41a0-9c55-9291e03d4b4b-ca-trust-extracted\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.268419 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.268399    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a431397d-d633-41a0-9c55-9291e03d4b4b-registry-certificates\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.269814 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.269790    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-registry-tls\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.269953 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.269937    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a431397d-d633-41a0-9c55-9291e03d4b4b-installation-pull-secrets\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.270019 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.269939    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a431397d-d633-41a0-9c55-9291e03d4b4b-image-registry-private-configuration\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.285844 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.285823    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 07:29:22.286453 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.286437    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75fbcd66cd-wf4ks"]
Oct 31 07:29:22.288583 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.288563    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a431397d-d633-41a0-9c55-9291e03d4b4b-trusted-ca\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.324294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.324259    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-bound-sa-token\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.324487 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.324386    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krwhl\" (UniqueName: \"kubernetes.io/projected/a431397d-d633-41a0-9c55-9291e03d4b4b-kube-api-access-krwhl\") pod \"image-registry-75fbcd66cd-wf4ks\" (UID: \"a431397d-d633-41a0-9c55-9291e03d4b4b\") " pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.380948 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.380925    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-c969t"]
Oct 31 07:29:22.382872 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.382857    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.433103 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.433080    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-94t84"
Oct 31 07:29:22.434500 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.434443    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-ppzxn\""
Oct 31 07:29:22.434665 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.434651    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 07:29:22.434714 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.434698    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 07:29:22.442092 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.442074    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-c969t"]
Oct 31 07:29:22.443839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.443823    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:22.469113 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.469086    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmbkh\" (UniqueName: \"kubernetes.io/projected/5416f823-2bb7-48db-b41a-c599c7af934c-kube-api-access-tmbkh\") pod \"downloads-6dd789c8b7-c969t\" (UID: \"5416f823-2bb7-48db-b41a-c599c7af934c\") " pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.469230 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.469148    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5416f823-2bb7-48db-b41a-c599c7af934c-tmp\") pod \"downloads-6dd789c8b7-c969t\" (UID: \"5416f823-2bb7-48db-b41a-c599c7af934c\") " pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.570201 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.570171    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5416f823-2bb7-48db-b41a-c599c7af934c-tmp\") pod \"downloads-6dd789c8b7-c969t\" (UID: \"5416f823-2bb7-48db-b41a-c599c7af934c\") " pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.570321 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.570220    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmbkh\" (UniqueName: \"kubernetes.io/projected/5416f823-2bb7-48db-b41a-c599c7af934c-kube-api-access-tmbkh\") pod \"downloads-6dd789c8b7-c969t\" (UID: \"5416f823-2bb7-48db-b41a-c599c7af934c\") " pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.570538 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.570518    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5416f823-2bb7-48db-b41a-c599c7af934c-tmp\") pod \"downloads-6dd789c8b7-c969t\" (UID: \"5416f823-2bb7-48db-b41a-c599c7af934c\") " pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.637998 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.637971    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmbkh\" (UniqueName: \"kubernetes.io/projected/5416f823-2bb7-48db-b41a-c599c7af934c-kube-api-access-tmbkh\") pod \"downloads-6dd789c8b7-c969t\" (UID: \"5416f823-2bb7-48db-b41a-c599c7af934c\") " pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.692107 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.692054    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:22.711532 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.711504    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5gj7z" event={"ID":"f4c82904-537f-49f8-bb3c-78fa7b00ccd6","Type":"ContainerStarted","Data":"a730e362f0c9cbfd57e248aaa01a6b46880f80bfffa3267b25016dd8b38ebc82"}
Oct 31 07:29:22.858535 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:22.858503    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75fbcd66cd-wf4ks"]
Oct 31 07:29:22.861747 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:22.861719    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda431397d_d633_41a0_9c55_9291e03d4b4b.slice/crio-930a4316489f9df0e32d0b4a0799fd42cbfeded641bbc153551e4bde7d0f0014 WatchSource:0}: Error finding container 930a4316489f9df0e32d0b4a0799fd42cbfeded641bbc153551e4bde7d0f0014: Status 404 returned error can't find the container with id 930a4316489f9df0e32d0b4a0799fd42cbfeded641bbc153551e4bde7d0f0014
Oct 31 07:29:23.055309 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:23.055281    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-c969t"]
Oct 31 07:29:23.058204 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:23.058180    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5416f823_2bb7_48db_b41a_c599c7af934c.slice/crio-a98d70435b4eabb9f21e402388491095071361f5d1f65493f1dbb6ea7547b4c7 WatchSource:0}: Error finding container a98d70435b4eabb9f21e402388491095071361f5d1f65493f1dbb6ea7547b4c7: Status 404 returned error can't find the container with id a98d70435b4eabb9f21e402388491095071361f5d1f65493f1dbb6ea7547b4c7
Oct 31 07:29:23.715622 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:23.715591    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks" event={"ID":"a431397d-d633-41a0-9c55-9291e03d4b4b","Type":"ContainerStarted","Data":"3349135e500f1086ca7f313cb6c0feab0b37ba979a2642b03a0c42c3c9be8497"}
Oct 31 07:29:23.716020 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:23.715641    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks" event={"ID":"a431397d-d633-41a0-9c55-9291e03d4b4b","Type":"ContainerStarted","Data":"930a4316489f9df0e32d0b4a0799fd42cbfeded641bbc153551e4bde7d0f0014"}
Oct 31 07:29:23.716020 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:23.715692    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:23.716701 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:23.716677    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-c969t" event={"ID":"5416f823-2bb7-48db-b41a-c599c7af934c","Type":"ContainerStarted","Data":"a98d70435b4eabb9f21e402388491095071361f5d1f65493f1dbb6ea7547b4c7"}
Oct 31 07:29:23.831791 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:23.831735    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks" podStartSLOduration=1.831721859 podStartE2EDuration="1.831721859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:23.831392463 +0000 UTC m=+14.333546253" watchObservedRunningTime="2026-10-31 07:29:23.831721859 +0000 UTC m=+14.333875653"
Oct 31 07:29:25.440203 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:25.440136    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zgfmt"
Oct 31 07:29:32.719778 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:32.719748    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mqsmg"
Oct 31 07:29:39.191620 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.191590    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rp2rq"]
Oct 31 07:29:39.208917 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.208896    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290225 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290199    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-root\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290338 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290237    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82rtr\" (UniqueName: \"kubernetes.io/projected/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-kube-api-access-82rtr\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290338 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290315    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-textfile\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290407 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290359    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-tls\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290407 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290383    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-sys\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290490 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290422    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-metrics-client-ca\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290529 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290456    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290529 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290516    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-wtmp\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.290603 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.290546    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.301105 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.301082    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c7ffn\""
Oct 31 07:29:39.301215 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.301144    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 07:29:39.309747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.309714    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 07:29:39.313488 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:29:39.313451    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:29:39.350559 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.350538    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 07:29:39.350683 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.350663    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 07:29:39.350732 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.350690    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 07:29:39.359225 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.359209    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 07:29:39.391141 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391118    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-textfile\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391213 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391143    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-tls\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391213 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391165    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-sys\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391213 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391182    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-metrics-client-ca\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391213 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391198    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391421 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391259    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-sys\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391421 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391307    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-wtmp\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391421 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391334    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391421 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391361    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-root\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.391421 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391400    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82rtr\" (UniqueName: \"kubernetes.io/projected/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-kube-api-access-82rtr\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.392056 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391502    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-wtmp\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.392056 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391526    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-textfile\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.392056 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391556    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-root\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.392056 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391863    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-metrics-client-ca\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.392056 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.391934    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.393390 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.393374    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-tls\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.393439 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.393415    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.446722 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.446676    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82rtr\" (UniqueName: \"kubernetes.io/projected/0755a07b-a0e0-4f53-b4aa-7013a5e2896f-kube-api-access-82rtr\") pod \"node-exporter-rp2rq\" (UID: \"0755a07b-a0e0-4f53-b4aa-7013a5e2896f\") " pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.638268 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.638232    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rp2rq"
Oct 31 07:29:39.770109 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.770025    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-c969t" event={"ID":"5416f823-2bb7-48db-b41a-c599c7af934c","Type":"ContainerStarted","Data":"d427d01d827d7c5e10975bb67ae5a2533b44eba4ec3089be9134f63ecafa4fbb"}
Oct 31 07:29:39.770532 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.770459    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:39.772006 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.771961    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2rq" event={"ID":"0755a07b-a0e0-4f53-b4aa-7013a5e2896f","Type":"ContainerStarted","Data":"6e59da33dfa8d8c448483238f91aee3523be0037cd2dca4b02b173d9b865c55f"}
Oct 31 07:29:39.786721 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.786702    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-c969t"
Oct 31 07:29:39.886224 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:39.886164    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-c969t" podStartSLOduration=1.7486038339999999 podStartE2EDuration="17.886151488s" totalImagesPullingTime="16.137547654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:39.885828177 +0000 UTC m=+30.387981966" watchObservedRunningTime="2026-10-31 07:29:39.886151488 +0000 UTC m=+30.388305278"
Oct 31 07:29:41.668909 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.668883    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7cl42"
Oct 31 07:29:41.748365 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.748334    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"]
Oct 31 07:29:41.771947 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.771919    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"]
Oct 31 07:29:41.772131 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.772111    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.823912 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.823856    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 31 07:29:41.823912 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.823876    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jnj4s\""
Oct 31 07:29:41.824074 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.823981    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bifrh3srtehtd\""
Oct 31 07:29:41.824074 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.824032    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 31 07:29:41.824186 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.824094    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 31 07:29:41.911674 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.911649    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.911785 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.911720    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef7fe687-91a4-4957-9526-b25b78f23309-metrics-client-ca\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.911921 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.911898    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.912004 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.911943    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef7fe687-91a4-4957-9526-b25b78f23309-tmp\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.912004 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.911979    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.912096 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.912070    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-tls\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.912138 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.912094    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjb8x\" (UniqueName: \"kubernetes.io/projected/ef7fe687-91a4-4957-9526-b25b78f23309-kube-api-access-kjb8x\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.912138 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.912117    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.912254 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.912137    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-grpc-tls\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:41.924170 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.924124    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 31 07:29:41.924704 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:41.924218    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 31 07:29:42.013547 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013520    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef7fe687-91a4-4957-9526-b25b78f23309-metrics-client-ca\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.013682 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013565    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.013682 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013590    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef7fe687-91a4-4957-9526-b25b78f23309-tmp\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.013682 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013624    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.013682 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013657    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-tls\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.013682 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013678    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjb8x\" (UniqueName: \"kubernetes.io/projected/ef7fe687-91a4-4957-9526-b25b78f23309-kube-api-access-kjb8x\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.014128 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013711    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.014128 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013727    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-grpc-tls\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.014128 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.013761    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.014964 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.014938    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef7fe687-91a4-4957-9526-b25b78f23309-metrics-client-ca\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.016293 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.016250    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef7fe687-91a4-4957-9526-b25b78f23309-tmp\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.019169 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.019145    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.019324 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.019209    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.021186 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.021160    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.021286 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.021196    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.021967 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.021943    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-grpc-tls\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.023206 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.023125    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ef7fe687-91a4-4957-9526-b25b78f23309-secret-thanos-querier-tls\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.071151 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.071123    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjb8x\" (UniqueName: \"kubernetes.io/projected/ef7fe687-91a4-4957-9526-b25b78f23309-kube-api-access-kjb8x\") pod \"thanos-querier-7876dcdfc5-dlqh7\" (UID: \"ef7fe687-91a4-4957-9526-b25b78f23309\") " pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.223649 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.223564    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:42.713916 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.713884    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"]
Oct 31 07:29:42.717170 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:42.717138    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef7fe687_91a4_4957_9526_b25b78f23309.slice/crio-c6234c243c42bf1be39e98e021d404b3730fe5f0c7504fa03bb4501900af1ae5 WatchSource:0}: Error finding container c6234c243c42bf1be39e98e021d404b3730fe5f0c7504fa03bb4501900af1ae5: Status 404 returned error can't find the container with id c6234c243c42bf1be39e98e021d404b3730fe5f0c7504fa03bb4501900af1ae5
Oct 31 07:29:42.786298 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.786269    4532 generic.go:356] "Generic (PLEG): container finished" podID="0755a07b-a0e0-4f53-b4aa-7013a5e2896f" containerID="83a443554b2d960bc964b328de60a2ce59fb80a32d6e79901c1eeedf1c478ca4" exitCode=0
Oct 31 07:29:42.786451 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.786344    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2rq" event={"ID":"0755a07b-a0e0-4f53-b4aa-7013a5e2896f","Type":"ContainerDied","Data":"83a443554b2d960bc964b328de60a2ce59fb80a32d6e79901c1eeedf1c478ca4"}
Oct 31 07:29:42.787781 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:42.787754    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" event={"ID":"ef7fe687-91a4-4957-9526-b25b78f23309","Type":"ContainerStarted","Data":"c6234c243c42bf1be39e98e021d404b3730fe5f0c7504fa03bb4501900af1ae5"}
Oct 31 07:29:43.795866 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:43.795826    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2rq" event={"ID":"0755a07b-a0e0-4f53-b4aa-7013a5e2896f","Type":"ContainerStarted","Data":"a3bf10d1c7dab1d3f3e79793dce3adcd47323e9a09c2fafa842c8e6e5a78f69d"}
Oct 31 07:29:43.796424 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:43.795892    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rp2rq" event={"ID":"0755a07b-a0e0-4f53-b4aa-7013a5e2896f","Type":"ContainerStarted","Data":"37736a58920cd5c477474b281dd0b5037c1675f9bd06e2beffdb3bb218b9a329"}
Oct 31 07:29:44.725091 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:44.725059    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75fbcd66cd-wf4ks"
Oct 31 07:29:44.940825 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:44.940765    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rp2rq" podStartSLOduration=3.69398817 podStartE2EDuration="5.940746918s" totalImagesPullingTime="2.155957419s" totalInitContainerRuntime="90.801329ms" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:43.913426017 +0000 UTC m=+34.415579807" watchObservedRunningTime="2026-10-31 07:29:44.940746918 +0000 UTC m=+35.442900710"
Oct 31 07:29:46.809698 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:46.809658    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" event={"ID":"ef7fe687-91a4-4957-9526-b25b78f23309","Type":"ContainerStarted","Data":"7bba679a87b7adb3ca0252ab4a9ef8ce54d0f734c7a9f444365674699af03357"}
Oct 31 07:29:46.810137 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:46.809766    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" event={"ID":"ef7fe687-91a4-4957-9526-b25b78f23309","Type":"ContainerStarted","Data":"35545be2527652d2f984078590735f97853e528140d531ddb7a8693202b3c939"}
Oct 31 07:29:46.810137 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:46.809794    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" event={"ID":"ef7fe687-91a4-4957-9526-b25b78f23309","Type":"ContainerStarted","Data":"807f4b21f4e19b018e4f26f4d61e9e8a927a4506c7cf2ea271800aeb41e758bd"}
Oct 31 07:29:48.232886 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.232345    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:29:48.267747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.267719    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.291915 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.291887    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:29:48.319787 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.319763    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 07:29:48.319881 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.319772    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 07:29:48.319881 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.319773    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 07:29:48.319881 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.319873    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fnhp8\""
Oct 31 07:29:48.339533 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.339511    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 07:29:48.339685 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.339555    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 07:29:48.339853 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.339837    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 07:29:48.339913 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.339837    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 07:29:48.339983 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.339925    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 07:29:48.339983 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.339951    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 07:29:48.340083 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.340011    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 31 07:29:48.340083 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.340075    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-55pgoq00lg91d\""
Oct 31 07:29:48.340185 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.340152    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 07:29:48.341553 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.341524    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 07:29:48.341553 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.341527    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 07:29:48.364091 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364064    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364190 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364118    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364190 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364143    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364318 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364246    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364318 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364277    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364419 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364325    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364498 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364418    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364498 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364445    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364498 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364489    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364604 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364536    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8vn6\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-kube-api-access-w8vn6\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364694 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364671    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364727 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364716    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364761 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364747    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364803 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364771    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364907 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364882    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.364956 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364928    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.365007 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.364986    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-config\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.365064 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.365037    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.365120 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.365069    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.365175 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.365115    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.439396 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.439363    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 07:29:48.439544 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.439457    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 07:29:48.467380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467351    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467523 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467388    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467523 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467406    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467523 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467423    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467523 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467439    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467735 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467575    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467735 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467631    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467735 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467654    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467735 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467682    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467735 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467706    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8vn6\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-kube-api-access-w8vn6\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467736    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467763    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467792    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467818    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467851    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467879    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467903    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-config\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467924    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467939    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.467981 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.467956    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.468478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.468411    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.468733 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.468707    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.469025 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.468996    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.469835 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.469813    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.469968 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.469920    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.470064 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.470040    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.470205 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.470182    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.471456 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.471429    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.471676 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.471653    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.471774 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.471680    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.472480 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.472144    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.472480 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.472428    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.472884 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.472857    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.472968 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.472863    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-config\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.473956 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.473932    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.474093 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.474069    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.474134 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.474073    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.474165 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.474129    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.474838 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.474819    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.526501 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.526483    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8vn6\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-kube-api-access-w8vn6\") pod \"prometheus-k8s-0\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.705531 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.705509    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:29:48.819415 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.819388    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" event={"ID":"ef7fe687-91a4-4957-9526-b25b78f23309","Type":"ContainerStarted","Data":"7ce485d98b2e5f94a8cd9abab2764db4a18011b053f9681bc61d04887cd49cc1"}
Oct 31 07:29:48.819580 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.819497    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" event={"ID":"ef7fe687-91a4-4957-9526-b25b78f23309","Type":"ContainerStarted","Data":"935ce9712d0f6d81d288f1d7c875d89c096aa8fec3f865665826cd966926ed12"}
Oct 31 07:29:48.819652 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.819637    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:48.819713 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.819689    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" event={"ID":"ef7fe687-91a4-4957-9526-b25b78f23309","Type":"ContainerStarted","Data":"7b143209bd30488b42f706a12bcdb2bbaf35810a51172ba6b30a454dc857e5d0"}
Oct 31 07:29:48.939373 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:48.939327    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7" podStartSLOduration=3.007409367 podStartE2EDuration="7.939312548s" totalImagesPullingTime="4.931903181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:48.937624646 +0000 UTC m=+39.439778436" watchObservedRunningTime="2026-10-31 07:29:48.939312548 +0000 UTC m=+39.441466338"
Oct 31 07:29:49.169650 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:49.169429    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:29:49.172492 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:29:49.172450    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f2f040_4115_425e_9d38_e50609fb42b7.slice/crio-ddc41eeb4068d3ca9fb3fe87aae14c54a55fae7e2c315945112c7dcc5348e178 WatchSource:0}: Error finding container ddc41eeb4068d3ca9fb3fe87aae14c54a55fae7e2c315945112c7dcc5348e178: Status 404 returned error can't find the container with id ddc41eeb4068d3ca9fb3fe87aae14c54a55fae7e2c315945112c7dcc5348e178
Oct 31 07:29:49.823800 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:49.823752    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerStarted","Data":"ddc41eeb4068d3ca9fb3fe87aae14c54a55fae7e2c315945112c7dcc5348e178"}
Oct 31 07:29:51.831152 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:51.831112    4532 generic.go:356] "Generic (PLEG): container finished" podID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a" exitCode=0
Oct 31 07:29:51.831662 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:51.831205    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"}
Oct 31 07:29:54.830795 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:54.830766    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7876dcdfc5-dlqh7"
Oct 31 07:29:56.853436 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:56.853403    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerStarted","Data":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"}
Oct 31 07:29:56.853875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:56.853529    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerStarted","Data":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"}
Oct 31 07:29:56.853875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:56.853578    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerStarted","Data":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"}
Oct 31 07:29:56.853875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:56.853620    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerStarted","Data":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"}
Oct 31 07:29:56.853875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:56.853658    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerStarted","Data":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"}
Oct 31 07:29:56.853875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:56.853699    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerStarted","Data":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"}
Oct 31 07:29:57.129955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:57.129873    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.045944741 podStartE2EDuration="9.129859131s" totalImagesPullingTime="6.062455249s" totalInitContainerRuntime="21.459141ms" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:57.129007398 +0000 UTC m=+47.631161189" watchObservedRunningTime="2026-10-31 07:29:57.129859131 +0000 UTC m=+47.632012922"
Oct 31 07:29:58.706333 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:29:58.706299    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:30:09.310028 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:30:09.309983    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:30:38.367676 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.367643    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-x4b5m-ps6h6"]
Oct 31 07:30:38.368074 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.367888    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6" podUID="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" containerName="insights-gathering" containerID="cri-o://7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60" gracePeriod=30
Oct 31 07:30:38.593145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.593122    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:30:38.627863 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.627802    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\") pod \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") "
Oct 31 07:30:38.627863 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.627837    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\") pod \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") "
Oct 31 07:30:38.627863 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.627861    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\") pod \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") "
Oct 31 07:30:38.628094 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.627885    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\" (UniqueName: \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\") pod \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\" (UID: \"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c\") "
Oct 31 07:30:38.628201 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.628174    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path" pod "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" (UID: "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:30:38.628261 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.628222    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle" pod "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" (UID: "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:38.628302 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.628250    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle" pod "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" (UID: "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:30:38.629955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.629930    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q" pod "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" (UID: "529a9ab9-cc1d-42a4-b34a-4cc09a38b81c"). InnerVolumeSpecName "kube-api-access-ktj2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:30:38.728717 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.728689    4532 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-service-ca-bundle\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:30:38.728717 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.728719    4532 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-trusted-ca-bundle\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:30:38.728904 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.728734    4532 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-archives-path\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:30:38.728904 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.728749    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktj2q\" (UniqueName: \"kubernetes.io/projected/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c-kube-api-access-ktj2q\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:30:38.989670 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.989597    4532 generic.go:356] "Generic (PLEG): container finished" podID="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" containerID="7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60" exitCode=2
Oct 31 07:30:38.989670 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.989634    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6" event={"ID":"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c","Type":"ContainerDied","Data":"7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60"}
Oct 31 07:30:38.989670 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.989647    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6"
Oct 31 07:30:38.989670 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.989668    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-x4b5m-ps6h6" event={"ID":"529a9ab9-cc1d-42a4-b34a-4cc09a38b81c","Type":"ContainerDied","Data":"8f75385dfa45c39fc8f946707654774d4350ae784dde318353f73c0598ab34a7"}
Oct 31 07:30:38.989957 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.989691    4532 scope.go:122] "RemoveContainer" containerID="7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60"
Oct 31 07:30:38.998294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.998276    4532 scope.go:122] "RemoveContainer" containerID="7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60"
Oct 31 07:30:38.998596 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:30:38.998574    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60\": container with ID starting with 7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60 not found: ID does not exist" containerID="7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60"
Oct 31 07:30:38.998657 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:38.998605    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60"} err="failed to get container status \"7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60\": rpc error: code = NotFound desc = could not find container \"7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60\": container with ID starting with 7e92ac2add6b4ef40505ba399c5397027fd4b9578fbc33f5f080ba9f29e92c60 not found: ID does not exist"
Oct 31 07:30:39.150350 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:39.150326    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-x4b5m-ps6h6"]
Oct 31 07:30:39.199663 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:39.199637    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-x4b5m-ps6h6"]
Oct 31 07:30:39.309741 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:30:39.309712    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:30:39.623706 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:39.623644    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" path="/var/lib/kubelet/pods/529a9ab9-cc1d-42a4-b34a-4cc09a38b81c/volumes"
Oct 31 07:30:51.505960 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:51.505919    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:30:51.524368 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:51.524340    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:30:52.045532 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:30:52.045504    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:09.309784 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:09.309750    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:31:11.179776 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.179745    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:31:11.180165 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.180039    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="prometheus" containerID="cri-o://3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351" gracePeriod=600
Oct 31 07:31:11.180165 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.180080    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901" gracePeriod=600
Oct 31 07:31:11.180165 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.180104    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-web" containerID="cri-o://ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5" gracePeriod=600
Oct 31 07:31:11.180165 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.180109    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="thanos-sidecar" containerID="cri-o://606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72" gracePeriod=600
Oct 31 07:31:11.180165 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.180128    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="config-reloader" containerID="cri-o://0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee" gracePeriod=600
Oct 31 07:31:11.180422 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.180071    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy" containerID="cri-o://52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827" gracePeriod=600
Oct 31 07:31:11.415242 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.415220    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:11.555352 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555327    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-metrics-client-certs\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555536 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555366    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555536 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555395    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-tls-assets\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-tls-assets\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555536 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555523    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-kube-rbac-proxy\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555718 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555574    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-thanos-prometheus-http-client-file\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555718 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555609    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-tls\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555718 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555639    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555718 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555671    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-grpc-tls\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555718 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555708    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-kubelet-serving-ca-bundle\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555977 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555744    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-web-config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-web-config\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555977 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555781    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-db\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555977 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555815    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-metrics-client-ca\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555977 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555850    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-config\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555977 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555885    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-kube-api-access-w8vn6\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-kube-api-access-w8vn6\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555977 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555927    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-1\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.555977 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.555970    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-2\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.556329 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.556008    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-serving-certs-ca-bundle\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.556329 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.556073    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-0\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.556329 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.556102    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-trusted-ca-bundle\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.556329 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.556132    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-config-out\") pod \"a7f2f040-4115-425e-9d38-e50609fb42b7\" (UID: \"a7f2f040-4115-425e-9d38-e50609fb42b7\") "
Oct 31 07:31:11.558864 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.558832    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-metrics-client-certs" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.559509 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.559308    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-kube-api-access-w8vn6" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "kube-api-access-w8vn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:31:11.559509 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.559384    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-config-out" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:31:11.559999 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.559962    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-kubelet-serving-ca-bundle" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:31:11.560188 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.560049    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-db" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.560559    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-metrics-client-ca" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.560708    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-thanos-sidecar-tls" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.560888    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-tls" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.560999    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-thanos-prometheus-http-client-file" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561111    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-kube-rbac-proxy" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561197    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-1" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561289    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-tls-assets" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561356    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-grpc-tls" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561390    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-2" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561399    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-kube-rbac-proxy-web" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.561659 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561631    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-trusted-ca-bundle" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:31:11.562141 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.561955    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-0" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:31:11.562368 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.562337    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-serving-certs-ca-bundle" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:31:11.562615 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.562598    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-config" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.571399 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.571376    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-web-config" pod "a7f2f040-4115-425e-9d38-e50609fb42b7" (UID: "a7f2f040-4115-425e-9d38-e50609fb42b7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:31:11.657033 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657009    4532 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657033 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657031    4532 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-tls-assets\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657041    4532 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-kube-rbac-proxy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657050    4532 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657059    4532 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657088    4532 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657098    4532 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-grpc-tls\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657107    4532 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657115    4532 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-web-config\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657124    4532 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-db\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657133    4532 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-metrics-client-ca\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657141    4532 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-config\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657150    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8vn6\" (UniqueName: \"kubernetes.io/projected/a7f2f040-4115-425e-9d38-e50609fb42b7-kube-api-access-w8vn6\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657157    4532 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657166    4532 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657174    4532 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657182    4532 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657190    4532 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7f2f040-4115-425e-9d38-e50609fb42b7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657200    4532 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7f2f040-4115-425e-9d38-e50609fb42b7-config-out\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:11.657520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:11.657208    4532 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7f2f040-4115-425e-9d38-e50609fb42b7-secret-metrics-client-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:31:12.096245 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096217    4532 generic.go:356] "Generic (PLEG): container finished" podID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901" exitCode=0
Oct 31 07:31:12.096245 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096242    4532 generic.go:356] "Generic (PLEG): container finished" podID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827" exitCode=0
Oct 31 07:31:12.096245 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096249    4532 generic.go:356] "Generic (PLEG): container finished" podID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5" exitCode=0
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096254    4532 generic.go:356] "Generic (PLEG): container finished" podID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72" exitCode=0
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096262    4532 generic.go:356] "Generic (PLEG): container finished" podID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee" exitCode=0
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096267    4532 generic.go:356] "Generic (PLEG): container finished" podID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351" exitCode=0
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096288    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096303    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"}
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096372    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"}
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096400    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"}
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096424    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"}
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096436    4532 scope.go:122] "RemoveContainer" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.096478 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096457    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"}
Oct 31 07:31:12.096916 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096493    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"}
Oct 31 07:31:12.096916 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.096524    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7f2f040-4115-425e-9d38-e50609fb42b7","Type":"ContainerDied","Data":"ddc41eeb4068d3ca9fb3fe87aae14c54a55fae7e2c315945112c7dcc5348e178"}
Oct 31 07:31:12.105144 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.105099    4532 scope.go:122] "RemoveContainer" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.112360 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.112345    4532 scope.go:122] "RemoveContainer" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.118321 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.118305    4532 scope.go:122] "RemoveContainer" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.124380 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.124353    4532 scope.go:122] "RemoveContainer" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.130525 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.130504    4532 scope.go:122] "RemoveContainer" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.137015 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.137002    4532 scope.go:122] "RemoveContainer" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.142922 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.142905    4532 scope.go:122] "RemoveContainer" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.143198 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:12.143180    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": container with ID starting with b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901 not found: ID does not exist" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.143256 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.143208    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"} err="failed to get container status \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": rpc error: code = NotFound desc = could not find container \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": container with ID starting with b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901 not found: ID does not exist"
Oct 31 07:31:12.143256 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.143227    4532 scope.go:122] "RemoveContainer" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.143499 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:12.143479    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": container with ID starting with 52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827 not found: ID does not exist" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.143573 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.143504    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"} err="failed to get container status \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": rpc error: code = NotFound desc = could not find container \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": container with ID starting with 52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827 not found: ID does not exist"
Oct 31 07:31:12.143573 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.143521    4532 scope.go:122] "RemoveContainer" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.143762 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:12.143749    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": container with ID starting with ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5 not found: ID does not exist" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.143800 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.143765    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"} err="failed to get container status \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": rpc error: code = NotFound desc = could not find container \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": container with ID starting with ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5 not found: ID does not exist"
Oct 31 07:31:12.143800 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.143776    4532 scope.go:122] "RemoveContainer" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.143989 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:12.143974    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": container with ID starting with 606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72 not found: ID does not exist" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.144023 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.143993    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"} err="failed to get container status \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": rpc error: code = NotFound desc = could not find container \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": container with ID starting with 606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72 not found: ID does not exist"
Oct 31 07:31:12.144023 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144005    4532 scope.go:122] "RemoveContainer" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.144212 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:12.144194    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": container with ID starting with 0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee not found: ID does not exist" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.144254 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144217    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"} err="failed to get container status \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": rpc error: code = NotFound desc = could not find container \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": container with ID starting with 0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee not found: ID does not exist"
Oct 31 07:31:12.144254 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144231    4532 scope.go:122] "RemoveContainer" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.144450 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:12.144436    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": container with ID starting with 3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351 not found: ID does not exist" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.144505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144454    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"} err="failed to get container status \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": rpc error: code = NotFound desc = could not find container \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": container with ID starting with 3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351 not found: ID does not exist"
Oct 31 07:31:12.144505 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144481    4532 scope.go:122] "RemoveContainer" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.144726 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:12.144709    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": container with ID starting with 88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a not found: ID does not exist" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.144780 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144730    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"} err="failed to get container status \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": rpc error: code = NotFound desc = could not find container \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": container with ID starting with 88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a not found: ID does not exist"
Oct 31 07:31:12.144780 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144742    4532 scope.go:122] "RemoveContainer" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.144926 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144907    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"} err="failed to get container status \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": rpc error: code = NotFound desc = could not find container \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": container with ID starting with b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901 not found: ID does not exist"
Oct 31 07:31:12.144973 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.144926    4532 scope.go:122] "RemoveContainer" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.145101 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145084    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"} err="failed to get container status \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": rpc error: code = NotFound desc = could not find container \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": container with ID starting with 52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827 not found: ID does not exist"
Oct 31 07:31:12.145154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145101    4532 scope.go:122] "RemoveContainer" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.145267 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145252    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"} err="failed to get container status \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": rpc error: code = NotFound desc = could not find container \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": container with ID starting with ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5 not found: ID does not exist"
Oct 31 07:31:12.145311 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145267    4532 scope.go:122] "RemoveContainer" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.145443 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145423    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"} err="failed to get container status \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": rpc error: code = NotFound desc = could not find container \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": container with ID starting with 606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72 not found: ID does not exist"
Oct 31 07:31:12.145535 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145444    4532 scope.go:122] "RemoveContainer" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.145656 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145640    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"} err="failed to get container status \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": rpc error: code = NotFound desc = could not find container \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": container with ID starting with 0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee not found: ID does not exist"
Oct 31 07:31:12.145698 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145656    4532 scope.go:122] "RemoveContainer" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.145859 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145841    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"} err="failed to get container status \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": rpc error: code = NotFound desc = could not find container \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": container with ID starting with 3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351 not found: ID does not exist"
Oct 31 07:31:12.145901 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.145859    4532 scope.go:122] "RemoveContainer" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.146061 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146043    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"} err="failed to get container status \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": rpc error: code = NotFound desc = could not find container \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": container with ID starting with 88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a not found: ID does not exist"
Oct 31 07:31:12.146104 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146062    4532 scope.go:122] "RemoveContainer" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.146292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146277    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"} err="failed to get container status \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": rpc error: code = NotFound desc = could not find container \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": container with ID starting with b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901 not found: ID does not exist"
Oct 31 07:31:12.146292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146291    4532 scope.go:122] "RemoveContainer" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.146550 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146534    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"} err="failed to get container status \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": rpc error: code = NotFound desc = could not find container \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": container with ID starting with 52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827 not found: ID does not exist"
Oct 31 07:31:12.146603 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146550    4532 scope.go:122] "RemoveContainer" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.146763 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146744    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"} err="failed to get container status \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": rpc error: code = NotFound desc = could not find container \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": container with ID starting with ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5 not found: ID does not exist"
Oct 31 07:31:12.146824 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146764    4532 scope.go:122] "RemoveContainer" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.147004 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.146990    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"} err="failed to get container status \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": rpc error: code = NotFound desc = could not find container \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": container with ID starting with 606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72 not found: ID does not exist"
Oct 31 07:31:12.147061 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147004    4532 scope.go:122] "RemoveContainer" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.147214 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147199    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"} err="failed to get container status \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": rpc error: code = NotFound desc = could not find container \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": container with ID starting with 0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee not found: ID does not exist"
Oct 31 07:31:12.147253 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147214    4532 scope.go:122] "RemoveContainer" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.147430 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147413    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"} err="failed to get container status \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": rpc error: code = NotFound desc = could not find container \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": container with ID starting with 3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351 not found: ID does not exist"
Oct 31 07:31:12.147503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147431    4532 scope.go:122] "RemoveContainer" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.147733 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147715    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"} err="failed to get container status \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": rpc error: code = NotFound desc = could not find container \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": container with ID starting with 88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a not found: ID does not exist"
Oct 31 07:31:12.147783 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147734    4532 scope.go:122] "RemoveContainer" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.147946 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147929    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"} err="failed to get container status \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": rpc error: code = NotFound desc = could not find container \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": container with ID starting with b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901 not found: ID does not exist"
Oct 31 07:31:12.147986 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.147947    4532 scope.go:122] "RemoveContainer" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.148197 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148181    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"} err="failed to get container status \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": rpc error: code = NotFound desc = could not find container \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": container with ID starting with 52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827 not found: ID does not exist"
Oct 31 07:31:12.148233 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148198    4532 scope.go:122] "RemoveContainer" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.148393 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148374    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"} err="failed to get container status \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": rpc error: code = NotFound desc = could not find container \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": container with ID starting with ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5 not found: ID does not exist"
Oct 31 07:31:12.148393 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148389    4532 scope.go:122] "RemoveContainer" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.148644 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148628    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"} err="failed to get container status \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": rpc error: code = NotFound desc = could not find container \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": container with ID starting with 606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72 not found: ID does not exist"
Oct 31 07:31:12.148644 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148643    4532 scope.go:122] "RemoveContainer" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.148875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148853    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"} err="failed to get container status \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": rpc error: code = NotFound desc = could not find container \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": container with ID starting with 0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee not found: ID does not exist"
Oct 31 07:31:12.148875 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.148876    4532 scope.go:122] "RemoveContainer" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.149120 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149101    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"} err="failed to get container status \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": rpc error: code = NotFound desc = could not find container \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": container with ID starting with 3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351 not found: ID does not exist"
Oct 31 07:31:12.149120 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149120    4532 scope.go:122] "RemoveContainer" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.149331 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149307    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"} err="failed to get container status \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": rpc error: code = NotFound desc = could not find container \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": container with ID starting with 88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a not found: ID does not exist"
Oct 31 07:31:12.149331 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149330    4532 scope.go:122] "RemoveContainer" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.149539 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149523    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"} err="failed to get container status \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": rpc error: code = NotFound desc = could not find container \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": container with ID starting with b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901 not found: ID does not exist"
Oct 31 07:31:12.149539 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149538    4532 scope.go:122] "RemoveContainer" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.149749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149731    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"} err="failed to get container status \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": rpc error: code = NotFound desc = could not find container \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": container with ID starting with 52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827 not found: ID does not exist"
Oct 31 07:31:12.149810 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149749    4532 scope.go:122] "RemoveContainer" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.149964 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149940    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"} err="failed to get container status \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": rpc error: code = NotFound desc = could not find container \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": container with ID starting with ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5 not found: ID does not exist"
Oct 31 07:31:12.149964 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.149963    4532 scope.go:122] "RemoveContainer" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.150158 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150138    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"} err="failed to get container status \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": rpc error: code = NotFound desc = could not find container \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": container with ID starting with 606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72 not found: ID does not exist"
Oct 31 07:31:12.150231 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150159    4532 scope.go:122] "RemoveContainer" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.150375 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150359    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"} err="failed to get container status \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": rpc error: code = NotFound desc = could not find container \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": container with ID starting with 0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee not found: ID does not exist"
Oct 31 07:31:12.150419 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150375    4532 scope.go:122] "RemoveContainer" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.150607 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150589    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"} err="failed to get container status \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": rpc error: code = NotFound desc = could not find container \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": container with ID starting with 3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351 not found: ID does not exist"
Oct 31 07:31:12.150651 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150609    4532 scope.go:122] "RemoveContainer" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.150818 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150803    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"} err="failed to get container status \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": rpc error: code = NotFound desc = could not find container \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": container with ID starting with 88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a not found: ID does not exist"
Oct 31 07:31:12.150863 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150819    4532 scope.go:122] "RemoveContainer" containerID="b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"
Oct 31 07:31:12.151019 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.150997    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901"} err="failed to get container status \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": rpc error: code = NotFound desc = could not find container \"b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901\": container with ID starting with b450e813f09c7fdfc5cfe749623e054b1ad146e71f3e82092c93c6edad884901 not found: ID does not exist"
Oct 31 07:31:12.151060 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151020    4532 scope.go:122] "RemoveContainer" containerID="52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"
Oct 31 07:31:12.151236 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151219    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827"} err="failed to get container status \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": rpc error: code = NotFound desc = could not find container \"52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827\": container with ID starting with 52e11be9943cff5c0eac778a177dbe2fc9dc1f6eaeff11a7689d88e1f5b29827 not found: ID does not exist"
Oct 31 07:31:12.151280 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151236    4532 scope.go:122] "RemoveContainer" containerID="ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"
Oct 31 07:31:12.151552 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151452    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5"} err="failed to get container status \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": rpc error: code = NotFound desc = could not find container \"ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5\": container with ID starting with ce4a775696533d653c4533779122368e36697d9372fa6ba0909ff9c86d5842d5 not found: ID does not exist"
Oct 31 07:31:12.151601 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151552    4532 scope.go:122] "RemoveContainer" containerID="606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"
Oct 31 07:31:12.151778 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151761    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72"} err="failed to get container status \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": rpc error: code = NotFound desc = could not find container \"606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72\": container with ID starting with 606d806a6c901f4db4ac7201c3e89399705da33d3c4a5734925b293478541a72 not found: ID does not exist"
Oct 31 07:31:12.151820 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151779    4532 scope.go:122] "RemoveContainer" containerID="0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"
Oct 31 07:31:12.151987 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151969    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee"} err="failed to get container status \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": rpc error: code = NotFound desc = could not find container \"0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee\": container with ID starting with 0308fdd443269dfca74e7c4d41176ec46dbfd55d85884fed69d7b7960cf972ee not found: ID does not exist"
Oct 31 07:31:12.152049 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.151988    4532 scope.go:122] "RemoveContainer" containerID="3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"
Oct 31 07:31:12.152197 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.152181    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351"} err="failed to get container status \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": rpc error: code = NotFound desc = could not find container \"3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351\": container with ID starting with 3b7b743ed3d8af497aed683e196782cb7f70d40a47d4137e84a484713bc15351 not found: ID does not exist"
Oct 31 07:31:12.152238 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.152197    4532 scope.go:122] "RemoveContainer" containerID="88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"
Oct 31 07:31:12.152388 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.152372    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a"} err="failed to get container status \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": rpc error: code = NotFound desc = could not find container \"88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a\": container with ID starting with 88533e8a2816e9e15b70f0f348deb9548de03b9f3752d75d57100d4403a7c55a not found: ID does not exist"
Oct 31 07:31:12.261177 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.261156    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:31:12.310983 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.310961    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:31:12.311590 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311572    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:31:12.311801 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311788    4532 cpu_manager.go:414] "containerMap: removing container" podUID="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" containerName="insights-gathering"
Oct 31 07:31:12.311839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311802    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" containerName="insights-gathering"
Oct 31 07:31:12.311839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311815    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="config-reloader"
Oct 31 07:31:12.311839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311820    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="config-reloader"
Oct 31 07:31:12.311839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311829    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="prometheus"
Oct 31 07:31:12.311839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311834    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="prometheus"
Oct 31 07:31:12.311839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311841    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311845    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311852    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-web"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311857    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-web"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311865    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="init-config-reloader"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311869    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="init-config-reloader"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311875    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-thanos"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311880    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-thanos"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311891    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="thanos-sidecar"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311896    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="thanos-sidecar"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311934    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-web"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311941    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy-thanos"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311950    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="thanos-sidecar"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311956    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="529a9ab9-cc1d-42a4-b34a-4cc09a38b81c" containerName="insights-gathering"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311961    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="config-reloader"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311967    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="prometheus"
Oct 31 07:31:12.312021 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.311972    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" containerName="kube-rbac-proxy"
Oct 31 07:31:12.314878 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.314864    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362169 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362117    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/edbce910-2fca-4c8d-b6bc-a83141840f04-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362169 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362145    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362287 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362167    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362287 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362231    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rlj4\" (UniqueName: \"kubernetes.io/projected/edbce910-2fca-4c8d-b6bc-a83141840f04-kube-api-access-5rlj4\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362362 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362310    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362362 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362349    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362429 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362400    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362459 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362423    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362459 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362459    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362589 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362523    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/edbce910-2fca-4c8d-b6bc-a83141840f04-config-out\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362589 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362555    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362589 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362570    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-config\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362684 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362604    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362684 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362622    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362684 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362644    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362777 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362680    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362777 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362731    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362777 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362758    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-web-config\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362871 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362776    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.362871 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.362817    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.367345 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367329    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 07:31:12.367749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367636    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 07:31:12.367749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367634    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-55pgoq00lg91d\""
Oct 31 07:31:12.367749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367659    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 07:31:12.367749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367688    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 07:31:12.367749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367682    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 07:31:12.367749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367678    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 07:31:12.368015 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367674    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 07:31:12.368077 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.367685    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 07:31:12.368734 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.368706    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 07:31:12.368837 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.368751    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 07:31:12.369284 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.369266    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fnhp8\""
Oct 31 07:31:12.369797 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.369769    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 07:31:12.369888 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.369864    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 31 07:31:12.369990 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.369973    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 07:31:12.417356 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.417339    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 07:31:12.463915 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.463893    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.463999 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.463920    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.463999 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.463938    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.463999 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.463963    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-web-config\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.463999 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.463990    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464013    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464048    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/edbce910-2fca-4c8d-b6bc-a83141840f04-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464093    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464139    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464171    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rlj4\" (UniqueName: \"kubernetes.io/projected/edbce910-2fca-4c8d-b6bc-a83141840f04-kube-api-access-5rlj4\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464198    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464229    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464262    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464291    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464313    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464388    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/edbce910-2fca-4c8d-b6bc-a83141840f04-config-out\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464450    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464419    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464539    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-config\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464546    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464572    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464599    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.464823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.464753    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.465170 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.465144    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.465610 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.465342    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.466910 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.466339    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.467236 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.467211    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.467457 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.467437    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/edbce910-2fca-4c8d-b6bc-a83141840f04-config-out\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.467787 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.467767    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-web-config\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.468309 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.468194    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.468309 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.468250    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.468801 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.468580    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/edbce910-2fca-4c8d-b6bc-a83141840f04-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.468801 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.468769    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 07:31:12.469017 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.468994    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.469090 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.469015    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-config\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.469394 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.469374    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.469697 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.469675    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.469828 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.469811    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.470259 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.470234    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/edbce910-2fca-4c8d-b6bc-a83141840f04-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.481449 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.481426    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/edbce910-2fca-4c8d-b6bc-a83141840f04-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.520338 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.520313    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rlj4\" (UniqueName: \"kubernetes.io/projected/edbce910-2fca-4c8d-b6bc-a83141840f04-kube-api-access-5rlj4\") pod \"prometheus-k8s-0\" (UID: \"edbce910-2fca-4c8d-b6bc-a83141840f04\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:12.535448 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.535424    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:31:12.629495 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:12.629426    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:13.049393 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:13.049368    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 07:31:13.051607 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:31:13.051584    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbce910_2fca_4c8d_b6bc_a83141840f04.slice/crio-682747793eeae7a080024fe1c4eab8b942a1aca9d2f0b87b0b3d636a28ac884c WatchSource:0}: Error finding container 682747793eeae7a080024fe1c4eab8b942a1aca9d2f0b87b0b3d636a28ac884c: Status 404 returned error can't find the container with id 682747793eeae7a080024fe1c4eab8b942a1aca9d2f0b87b0b3d636a28ac884c
Oct 31 07:31:13.102270 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:13.102245    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerStarted","Data":"682747793eeae7a080024fe1c4eab8b942a1aca9d2f0b87b0b3d636a28ac884c"}
Oct 31 07:31:13.623498 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:13.623450    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7f2f040-4115-425e-9d38-e50609fb42b7" path="/var/lib/kubelet/pods/a7f2f040-4115-425e-9d38-e50609fb42b7/volumes"
Oct 31 07:31:14.106607 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:14.106579    4532 generic.go:356] "Generic (PLEG): container finished" podID="edbce910-2fca-4c8d-b6bc-a83141840f04" containerID="64880c1e44de797a2f2990b17140e1beee7189f3fd9cff1517f11adb29f3029f" exitCode=0
Oct 31 07:31:14.106752 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:14.106612    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerDied","Data":"64880c1e44de797a2f2990b17140e1beee7189f3fd9cff1517f11adb29f3029f"}
Oct 31 07:31:15.112767 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:15.112733    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerStarted","Data":"3d0df94f2422e24ac88a02b9b45e1ca84fe29aad3be2daf4e4c291a183714797"}
Oct 31 07:31:15.113211 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:15.112838    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerStarted","Data":"026857815e7ecac184a9fb2f030207fae6a6cc6d2e7d15c3109ad15a99c9a639"}
Oct 31 07:31:15.113211 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:15.112893    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerStarted","Data":"82eee49350cab87ffbf9ee497f613fe83f74c1aacd90f48e20d6b19acd07a331"}
Oct 31 07:31:15.113211 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:15.112937    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerStarted","Data":"0a1905fc01f702e9df68eb905e02a96ae2845b5a7cdfb83e4daccdfa806e4f5e"}
Oct 31 07:31:15.113211 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:15.112978    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerStarted","Data":"e2499e9b642d7cf6fca30dea13d7c34fb3466c673d0eb6b1baa4a167e62e2b6d"}
Oct 31 07:31:15.113211 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:15.113015    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"edbce910-2fca-4c8d-b6bc-a83141840f04","Type":"ContainerStarted","Data":"052dac39afeac48a4c93ed62c76e0bf8e5602dd613b1ae4e481adec940c5c353"}
Oct 31 07:31:15.234218 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:15.234161    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.222551911 podStartE2EDuration="3.234143946s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.592035ms" isStatefulPod=false podCreationTimestamp="2026-10-31 07:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:31:15.233079626 +0000 UTC m=+125.735233415" watchObservedRunningTime="2026-10-31 07:31:15.234143946 +0000 UTC m=+125.736297738"
Oct 31 07:31:17.629908 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:31:17.629877    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:31:39.310562 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:31:39.310483    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:32:09.310140 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:32:09.310102    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:32:12.630263 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:32:12.630230    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:32:12.645366 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:32:12.645342    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:32:13.311615 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:32:13.311588    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 07:32:39.310234 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:32:39.310200    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:33:09.310243 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:33:09.310205    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:33:39.310077 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:33:39.310041    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:34:09.309505 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:34:09.309444    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:34:39.310636 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:34:39.310557    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:35:09.310413 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:35:09.310372    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:35:39.310640 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:35:39.310603    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:36:09.310708 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:36:09.310671    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:36:23.178569 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.178537    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-xvr88"]
Oct 31 07:36:23.181611 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.181595    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.233510 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.233484    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 31 07:36:23.282937 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.282912    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 31 07:36:23.282937 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.282925    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-z98hh\""
Oct 31 07:36:23.283940 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.283921    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-xvr88"]
Oct 31 07:36:23.339421 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.339386    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t86bq\" (UniqueName: \"kubernetes.io/projected/60dfdbed-520d-4101-8974-71fa3c005686-kube-api-access-t86bq\") pod \"cert-manager-597b95cb84-xvr88\" (UID: \"60dfdbed-520d-4101-8974-71fa3c005686\") " pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.339548 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.339441    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60dfdbed-520d-4101-8974-71fa3c005686-bound-sa-token\") pod \"cert-manager-597b95cb84-xvr88\" (UID: \"60dfdbed-520d-4101-8974-71fa3c005686\") " pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.440260 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.440192    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t86bq\" (UniqueName: \"kubernetes.io/projected/60dfdbed-520d-4101-8974-71fa3c005686-kube-api-access-t86bq\") pod \"cert-manager-597b95cb84-xvr88\" (UID: \"60dfdbed-520d-4101-8974-71fa3c005686\") " pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.440260 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.440242    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60dfdbed-520d-4101-8974-71fa3c005686-bound-sa-token\") pod \"cert-manager-597b95cb84-xvr88\" (UID: \"60dfdbed-520d-4101-8974-71fa3c005686\") " pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.496217 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.496195    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t86bq\" (UniqueName: \"kubernetes.io/projected/60dfdbed-520d-4101-8974-71fa3c005686-kube-api-access-t86bq\") pod \"cert-manager-597b95cb84-xvr88\" (UID: \"60dfdbed-520d-4101-8974-71fa3c005686\") " pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.496330 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.496284    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60dfdbed-520d-4101-8974-71fa3c005686-bound-sa-token\") pod \"cert-manager-597b95cb84-xvr88\" (UID: \"60dfdbed-520d-4101-8974-71fa3c005686\") " pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.505099 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.505080    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-xvr88"
Oct 31 07:36:23.868749 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.868722    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-xvr88"]
Oct 31 07:36:23.871501 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:36:23.871473    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60dfdbed_520d_4101_8974_71fa3c005686.slice/crio-bedddf0e35872a26eb83a5be7228ef3f3d112d1413def3f755c67029146f82b3 WatchSource:0}: Error finding container bedddf0e35872a26eb83a5be7228ef3f3d112d1413def3f755c67029146f82b3: Status 404 returned error can't find the container with id bedddf0e35872a26eb83a5be7228ef3f3d112d1413def3f755c67029146f82b3
Oct 31 07:36:23.873258 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:23.873237    4532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:36:24.088569 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:24.088542    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-xvr88" event={"ID":"60dfdbed-520d-4101-8974-71fa3c005686","Type":"ContainerStarted","Data":"bedddf0e35872a26eb83a5be7228ef3f3d112d1413def3f755c67029146f82b3"}
Oct 31 07:36:28.107141 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:28.107104    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-xvr88" event={"ID":"60dfdbed-520d-4101-8974-71fa3c005686","Type":"ContainerStarted","Data":"29323630328d90ebd692f24505560d6908d8032defecaea672137fda74e96e8d"}
Oct 31 07:36:28.219372 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:28.219274    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-xvr88" podStartSLOduration=1.102824113 podStartE2EDuration="5.219258781s" totalImagesPullingTime="4.116434668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:36:28.218760075 +0000 UTC m=+438.720913872" watchObservedRunningTime="2026-10-31 07:36:28.219258781 +0000 UTC m=+438.721412572"
Oct 31 07:36:36.608438 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.608403    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"]
Oct 31 07:36:36.611698 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.611682    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.631029 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.631004    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/649ba68b-f08f-40a3-9e39-29b080906845-metrics-cert\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.631153 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.631133    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/649ba68b-f08f-40a3-9e39-29b080906845-manager-config\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.631209 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.631182    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drvh\" (UniqueName: \"kubernetes.io/projected/649ba68b-f08f-40a3-9e39-29b080906845-kube-api-access-8drvh\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.631364 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.631347    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/649ba68b-f08f-40a3-9e39-29b080906845-cert\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.663564 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.663545    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 31 07:36:36.663658 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.663637    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 31 07:36:36.712903 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.712875    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 31 07:36:36.713013 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.712912    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6jm8r\""
Oct 31 07:36:36.713013 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.712972    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 31 07:36:36.713013 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.712970    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 31 07:36:36.713885 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.713870    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"]
Oct 31 07:36:36.732407 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.732386    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/649ba68b-f08f-40a3-9e39-29b080906845-metrics-cert\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.732520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.732431    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/649ba68b-f08f-40a3-9e39-29b080906845-manager-config\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.732520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.732455    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drvh\" (UniqueName: \"kubernetes.io/projected/649ba68b-f08f-40a3-9e39-29b080906845-kube-api-access-8drvh\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.732520 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.732495    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/649ba68b-f08f-40a3-9e39-29b080906845-cert\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.733041 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.733018    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/649ba68b-f08f-40a3-9e39-29b080906845-manager-config\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.735286 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.735261    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/649ba68b-f08f-40a3-9e39-29b080906845-cert\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.735363 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.735302    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/649ba68b-f08f-40a3-9e39-29b080906845-metrics-cert\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.787890 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.787866    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drvh\" (UniqueName: \"kubernetes.io/projected/649ba68b-f08f-40a3-9e39-29b080906845-kube-api-access-8drvh\") pod \"lws-controller-manager-64dc655d5d-sgm9d\" (UID: \"649ba68b-f08f-40a3-9e39-29b080906845\") " pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:36.922663 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:36.922609    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:37.289988 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:37.289963    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"]
Oct 31 07:36:37.292717 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:36:37.292689    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod649ba68b_f08f_40a3_9e39_29b080906845.slice/crio-f57bf22e3f2a79eed233eba4f1373183d45d9dcd0095004068b97cd584e1bea9 WatchSource:0}: Error finding container f57bf22e3f2a79eed233eba4f1373183d45d9dcd0095004068b97cd584e1bea9: Status 404 returned error can't find the container with id f57bf22e3f2a79eed233eba4f1373183d45d9dcd0095004068b97cd584e1bea9
Oct 31 07:36:38.140454 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:38.140422    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d" event={"ID":"649ba68b-f08f-40a3-9e39-29b080906845","Type":"ContainerStarted","Data":"f57bf22e3f2a79eed233eba4f1373183d45d9dcd0095004068b97cd584e1bea9"}
Oct 31 07:36:39.309354 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:36:39.309316    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:36:42.155220 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:42.155182    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d" event={"ID":"649ba68b-f08f-40a3-9e39-29b080906845","Type":"ContainerStarted","Data":"2447b2105d364d434516e5620ba1307e25bed4a63fcdca4acb34f9f117b0a989"}
Oct 31 07:36:42.155695 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:42.155304    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:36:42.267939 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:42.267889    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d" podStartSLOduration=2.426536147 podStartE2EDuration="6.267876875s" totalImagesPullingTime="3.841340728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:36:42.266885584 +0000 UTC m=+452.769039407" watchObservedRunningTime="2026-10-31 07:36:42.267876875 +0000 UTC m=+452.770030665"
Oct 31 07:36:53.160573 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:36:53.160539    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64dc655d5d-sgm9d"
Oct 31 07:37:09.310983 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:37:09.310951    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:37:16.273514 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.273478    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-qxqkz"]
Oct 31 07:37:16.278547 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.278522    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz"
Oct 31 07:37:16.380254 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.380231    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kdmbv\""
Oct 31 07:37:16.380390 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.380282    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 31 07:37:16.380449 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.380388    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 31 07:37:16.381602 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.381581    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-qxqkz"]
Oct 31 07:37:16.406798 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.406775    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgc6r\" (UniqueName: \"kubernetes.io/projected/2e339cac-5d87-48ed-ae4d-a41a58cdb012-kube-api-access-fgc6r\") pod \"authorino-operator-66498fdf6c-qxqkz\" (UID: \"2e339cac-5d87-48ed-ae4d-a41a58cdb012\") " pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz"
Oct 31 07:37:16.507221 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.507194    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgc6r\" (UniqueName: \"kubernetes.io/projected/2e339cac-5d87-48ed-ae4d-a41a58cdb012-kube-api-access-fgc6r\") pod \"authorino-operator-66498fdf6c-qxqkz\" (UID: \"2e339cac-5d87-48ed-ae4d-a41a58cdb012\") " pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz"
Oct 31 07:37:16.563536 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.563507    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgc6r\" (UniqueName: \"kubernetes.io/projected/2e339cac-5d87-48ed-ae4d-a41a58cdb012-kube-api-access-fgc6r\") pod \"authorino-operator-66498fdf6c-qxqkz\" (UID: \"2e339cac-5d87-48ed-ae4d-a41a58cdb012\") " pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz"
Oct 31 07:37:16.589511 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.589479    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz"
Oct 31 07:37:16.957996 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:16.957928    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-qxqkz"]
Oct 31 07:37:16.961247 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:37:16.961217    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e339cac_5d87_48ed_ae4d_a41a58cdb012.slice/crio-4d32d57859826cfc350290aed6f5edffb24a17cf4b4dd58af06da50df555b5c2 WatchSource:0}: Error finding container 4d32d57859826cfc350290aed6f5edffb24a17cf4b4dd58af06da50df555b5c2: Status 404 returned error can't find the container with id 4d32d57859826cfc350290aed6f5edffb24a17cf4b4dd58af06da50df555b5c2
Oct 31 07:37:17.283219 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:17.283183    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz" event={"ID":"2e339cac-5d87-48ed-ae4d-a41a58cdb012","Type":"ContainerStarted","Data":"4d32d57859826cfc350290aed6f5edffb24a17cf4b4dd58af06da50df555b5c2"}
Oct 31 07:37:21.298089 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:21.298053    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz" event={"ID":"2e339cac-5d87-48ed-ae4d-a41a58cdb012","Type":"ContainerStarted","Data":"3f4dfa6e95b1a3f69e7320a98ee54e1fec46155c5e50783dfae2d9254af35432"}
Oct 31 07:37:21.298575 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:21.298198    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz"
Oct 31 07:37:21.412435 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:21.412388    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz" podStartSLOduration=2.044965378 podStartE2EDuration="5.412368571s" totalImagesPullingTime="3.367403193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:37:21.411253465 +0000 UTC m=+491.913407256" watchObservedRunningTime="2026-10-31 07:37:21.412368571 +0000 UTC m=+491.914522361"
Oct 31 07:37:32.304171 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:37:32.304143    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-66498fdf6c-qxqkz"
Oct 31 07:37:39.310508 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:37:39.310452    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:38:09.309283 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:38:09.309245    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:38:14.696899 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:14.696864    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fm5rl"]
Oct 31 07:38:14.701829 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:14.701808    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fm5rl"
Oct 31 07:38:14.753944 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:14.753922    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fm5rl"]
Oct 31 07:38:14.802224 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:14.802204    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tmh6h\""
Oct 31 07:38:14.802348 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:14.802312    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbtvw\" (UniqueName: \"kubernetes.io/projected/0949220c-ff17-4039-ab6a-55d187f1a174-kube-api-access-sbtvw\") pod \"authorino-57f7f8659f-fm5rl\" (UID: \"0949220c-ff17-4039-ab6a-55d187f1a174\") " pod="kuadrant-system/authorino-57f7f8659f-fm5rl"
Oct 31 07:38:14.903587 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:14.903557    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbtvw\" (UniqueName: \"kubernetes.io/projected/0949220c-ff17-4039-ab6a-55d187f1a174-kube-api-access-sbtvw\") pod \"authorino-57f7f8659f-fm5rl\" (UID: \"0949220c-ff17-4039-ab6a-55d187f1a174\") " pod="kuadrant-system/authorino-57f7f8659f-fm5rl"
Oct 31 07:38:14.959311 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:14.959258    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbtvw\" (UniqueName: \"kubernetes.io/projected/0949220c-ff17-4039-ab6a-55d187f1a174-kube-api-access-sbtvw\") pod \"authorino-57f7f8659f-fm5rl\" (UID: \"0949220c-ff17-4039-ab6a-55d187f1a174\") " pod="kuadrant-system/authorino-57f7f8659f-fm5rl"
Oct 31 07:38:15.013343 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:15.013320    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fm5rl"
Oct 31 07:38:15.378340 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:15.378310    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fm5rl"]
Oct 31 07:38:15.381299 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:38:15.381262    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0949220c_ff17_4039_ab6a_55d187f1a174.slice/crio-8d9ae4ef243f8b4ed48709aa7d5e9d83c80c4798d6b0fa513b2c41c3f33cfa98 WatchSource:0}: Error finding container 8d9ae4ef243f8b4ed48709aa7d5e9d83c80c4798d6b0fa513b2c41c3f33cfa98: Status 404 returned error can't find the container with id 8d9ae4ef243f8b4ed48709aa7d5e9d83c80c4798d6b0fa513b2c41c3f33cfa98
Oct 31 07:38:15.483012 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:15.482985    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fm5rl" event={"ID":"0949220c-ff17-4039-ab6a-55d187f1a174","Type":"ContainerStarted","Data":"8d9ae4ef243f8b4ed48709aa7d5e9d83c80c4798d6b0fa513b2c41c3f33cfa98"}
Oct 31 07:38:18.495896 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:18.495860    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fm5rl" event={"ID":"0949220c-ff17-4039-ab6a-55d187f1a174","Type":"ContainerStarted","Data":"cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c"}
Oct 31 07:38:18.607177 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:18.607095    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-fm5rl" podStartSLOduration=1.6644796720000001 podStartE2EDuration="4.607082132s" totalImagesPullingTime="2.94260246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:38:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:38:18.605510636 +0000 UTC m=+549.107664427" watchObservedRunningTime="2026-10-31 07:38:18.607082132 +0000 UTC m=+549.109235922"
Oct 31 07:38:21.259951 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.259916    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fm5rl"]
Oct 31 07:38:21.260434 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.260086    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-fm5rl" podUID="0949220c-ff17-4039-ab6a-55d187f1a174" containerName="authorino" containerID="cri-o://cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c" gracePeriod=30
Oct 31 07:38:21.490718 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.490695    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fm5rl"
Oct 31 07:38:21.507227 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.507201    4532 generic.go:356] "Generic (PLEG): container finished" podID="0949220c-ff17-4039-ab6a-55d187f1a174" containerID="cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c" exitCode=0
Oct 31 07:38:21.507322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.507233    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-fm5rl"
Oct 31 07:38:21.507322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.507270    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fm5rl" event={"ID":"0949220c-ff17-4039-ab6a-55d187f1a174","Type":"ContainerDied","Data":"cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c"}
Oct 31 07:38:21.507322 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.507315    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-fm5rl" event={"ID":"0949220c-ff17-4039-ab6a-55d187f1a174","Type":"ContainerDied","Data":"8d9ae4ef243f8b4ed48709aa7d5e9d83c80c4798d6b0fa513b2c41c3f33cfa98"}
Oct 31 07:38:21.507419 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.507332    4532 scope.go:122] "RemoveContainer" containerID="cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c"
Oct 31 07:38:21.514357 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.514341    4532 scope.go:122] "RemoveContainer" containerID="cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c"
Oct 31 07:38:21.514633 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:38:21.514615    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c\": container with ID starting with cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c not found: ID does not exist" containerID="cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c"
Oct 31 07:38:21.514706 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.514648    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c"} err="failed to get container status \"cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c\": rpc error: code = NotFound desc = could not find container \"cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c\": container with ID starting with cf10a32df34ebc7414ba214f434f2ed18cc79bd6f8aeecce013c3c59b6e5384c not found: ID does not exist"
Oct 31 07:38:21.553418 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.553398    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0949220c-ff17-4039-ab6a-55d187f1a174-kube-api-access-sbtvw\" (UniqueName: \"kubernetes.io/projected/0949220c-ff17-4039-ab6a-55d187f1a174-kube-api-access-sbtvw\") pod \"0949220c-ff17-4039-ab6a-55d187f1a174\" (UID: \"0949220c-ff17-4039-ab6a-55d187f1a174\") "
Oct 31 07:38:21.555341 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.555320    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0949220c-ff17-4039-ab6a-55d187f1a174-kube-api-access-sbtvw" pod "0949220c-ff17-4039-ab6a-55d187f1a174" (UID: "0949220c-ff17-4039-ab6a-55d187f1a174"). InnerVolumeSpecName "kube-api-access-sbtvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:38:21.654122 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.654092    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbtvw\" (UniqueName: \"kubernetes.io/projected/0949220c-ff17-4039-ab6a-55d187f1a174-kube-api-access-sbtvw\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:38:21.918567 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.918541    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fm5rl"]
Oct 31 07:38:21.922629 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:21.922600    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-fm5rl"]
Oct 31 07:38:23.622750 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:38:23.622717    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0949220c-ff17-4039-ab6a-55d187f1a174" path="/var/lib/kubelet/pods/0949220c-ff17-4039-ab6a-55d187f1a174/volumes"
Oct 31 07:38:39.310176 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:38:39.310069    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:39:09.310200 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:39:09.310165    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:39:12.574859 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.574823    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557c8f8584-nxqhs"]
Oct 31 07:39:12.575261 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.575107    4532 cpu_manager.go:414] "containerMap: removing container" podUID="0949220c-ff17-4039-ab6a-55d187f1a174" containerName="authorino"
Oct 31 07:39:12.575261 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.575119    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0949220c-ff17-4039-ab6a-55d187f1a174" containerName="authorino"
Oct 31 07:39:12.575261 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.575188    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="0949220c-ff17-4039-ab6a-55d187f1a174" containerName="authorino"
Oct 31 07:39:12.578287 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.578270    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:12.636616 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.636596    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 31 07:39:12.636834 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.636724    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tmh6h\""
Oct 31 07:39:12.682666 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.682642    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557c8f8584-nxqhs"]
Oct 31 07:39:12.712594 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.712570    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd7ms\" (UniqueName: \"kubernetes.io/projected/6a9dac1c-1c08-4cc4-84f9-b521f779b7ed-kube-api-access-gd7ms\") pod \"authorino-557c8f8584-nxqhs\" (UID: \"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed\") " pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:12.712722 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.712620    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a9dac1c-1c08-4cc4-84f9-b521f779b7ed-tls-cert\") pod \"authorino-557c8f8584-nxqhs\" (UID: \"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed\") " pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:12.813848 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.813820    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a9dac1c-1c08-4cc4-84f9-b521f779b7ed-tls-cert\") pod \"authorino-557c8f8584-nxqhs\" (UID: \"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed\") " pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:12.813966 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.813869    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd7ms\" (UniqueName: \"kubernetes.io/projected/6a9dac1c-1c08-4cc4-84f9-b521f779b7ed-kube-api-access-gd7ms\") pod \"authorino-557c8f8584-nxqhs\" (UID: \"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed\") " pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:12.816274 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.816247    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6a9dac1c-1c08-4cc4-84f9-b521f779b7ed-tls-cert\") pod \"authorino-557c8f8584-nxqhs\" (UID: \"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed\") " pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:12.869075 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.869022    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd7ms\" (UniqueName: \"kubernetes.io/projected/6a9dac1c-1c08-4cc4-84f9-b521f779b7ed-kube-api-access-gd7ms\") pod \"authorino-557c8f8584-nxqhs\" (UID: \"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed\") " pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:12.888857 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:12.888838    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557c8f8584-nxqhs"
Oct 31 07:39:13.256455 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:13.256388    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557c8f8584-nxqhs"]
Oct 31 07:39:13.259726 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:39:13.259701    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a9dac1c_1c08_4cc4_84f9_b521f779b7ed.slice/crio-523d8df2b0a8c68f2b1675fd3783a2de91bb33d6cfe52bb53519e511af23d08b WatchSource:0}: Error finding container 523d8df2b0a8c68f2b1675fd3783a2de91bb33d6cfe52bb53519e511af23d08b: Status 404 returned error can't find the container with id 523d8df2b0a8c68f2b1675fd3783a2de91bb33d6cfe52bb53519e511af23d08b
Oct 31 07:39:13.682843 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:13.682808    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557c8f8584-nxqhs" event={"ID":"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed","Type":"ContainerStarted","Data":"523d8df2b0a8c68f2b1675fd3783a2de91bb33d6cfe52bb53519e511af23d08b"}
Oct 31 07:39:14.687867 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:14.687831    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557c8f8584-nxqhs" event={"ID":"6a9dac1c-1c08-4cc4-84f9-b521f779b7ed","Type":"ContainerStarted","Data":"2e8de32d32ccb114e4336001a7c747c17b1fa3f347bac1ada616fefb9c7afd3e"}
Oct 31 07:39:14.798924 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:14.798876    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-557c8f8584-nxqhs" podStartSLOduration=1.573112351 podStartE2EDuration="2.798863185s" totalImagesPullingTime="1.225750834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:39:14.798577024 +0000 UTC m=+605.300730815" watchObservedRunningTime="2026-10-31 07:39:14.798863185 +0000 UTC m=+605.301016977"
Oct 31 07:39:39.310293 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:39:39.310252    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:39:44.871650 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:44.871618    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"]
Oct 31 07:39:44.874889 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:44.874873    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:44.958303 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:44.958274    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca2c150-8e74-4090-9b60-9608f8d46ff7-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-vrh4x\" (UID: \"5ca2c150-8e74-4090-9b60-9608f8d46ff7\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:44.958727 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:44.958336    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m82hr\" (UniqueName: \"kubernetes.io/projected/5ca2c150-8e74-4090-9b60-9608f8d46ff7-kube-api-access-m82hr\") pod \"llmisvc-controller-manager-86c86ffb6b-vrh4x\" (UID: \"5ca2c150-8e74-4090-9b60-9608f8d46ff7\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:44.975389 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:44.975363    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Oct 31 07:39:44.975525 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:44.975412    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-tngh5\""
Oct 31 07:39:44.976596 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:44.976576    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"]
Oct 31 07:39:45.024726 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.024700    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 31 07:39:45.024726 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.024703    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 31 07:39:45.060053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.060029    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m82hr\" (UniqueName: \"kubernetes.io/projected/5ca2c150-8e74-4090-9b60-9608f8d46ff7-kube-api-access-m82hr\") pod \"llmisvc-controller-manager-86c86ffb6b-vrh4x\" (UID: \"5ca2c150-8e74-4090-9b60-9608f8d46ff7\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:45.060154 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.060131    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca2c150-8e74-4090-9b60-9608f8d46ff7-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-vrh4x\" (UID: \"5ca2c150-8e74-4090-9b60-9608f8d46ff7\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:45.062425 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.062406    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ca2c150-8e74-4090-9b60-9608f8d46ff7-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-vrh4x\" (UID: \"5ca2c150-8e74-4090-9b60-9608f8d46ff7\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:45.116161 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.116138    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m82hr\" (UniqueName: \"kubernetes.io/projected/5ca2c150-8e74-4090-9b60-9608f8d46ff7-kube-api-access-m82hr\") pod \"llmisvc-controller-manager-86c86ffb6b-vrh4x\" (UID: \"5ca2c150-8e74-4090-9b60-9608f8d46ff7\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:45.187246 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.187199    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:45.550544 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.550514    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"]
Oct 31 07:39:45.553310 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:39:45.553284    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca2c150_8e74_4090_9b60_9608f8d46ff7.slice/crio-2d23ab0e4eb1b762df764b0ebaf3598deedfb82d21636fc0098a2ff80d373e94 WatchSource:0}: Error finding container 2d23ab0e4eb1b762df764b0ebaf3598deedfb82d21636fc0098a2ff80d373e94: Status 404 returned error can't find the container with id 2d23ab0e4eb1b762df764b0ebaf3598deedfb82d21636fc0098a2ff80d373e94
Oct 31 07:39:45.786182 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:45.786149    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x" event={"ID":"5ca2c150-8e74-4090-9b60-9608f8d46ff7","Type":"ContainerStarted","Data":"2d23ab0e4eb1b762df764b0ebaf3598deedfb82d21636fc0098a2ff80d373e94"}
Oct 31 07:39:51.809831 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:51.809798    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x" event={"ID":"5ca2c150-8e74-4090-9b60-9608f8d46ff7","Type":"ContainerStarted","Data":"4d4a44bad3dee9b486a275fae18e74db02ca7f788cc08c02f1945610005867ff"}
Oct 31 07:39:51.810179 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:51.809979    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:39:51.921894 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:39:51.921848    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x" podStartSLOduration=2.054568694 podStartE2EDuration="7.921831859s" totalImagesPullingTime="5.867263165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:39:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:39:51.920841162 +0000 UTC m=+642.422994951" watchObservedRunningTime="2026-10-31 07:39:51.921831859 +0000 UTC m=+642.423985652"
Oct 31 07:40:09.310612 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:40:09.310571    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:40:22.845537 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:40:22.844703    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-vrh4x"
Oct 31 07:40:39.309518 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:40:39.309484    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:41:09.309866 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:41:09.309829    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:41:39.310291 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:41:39.310250    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:42:09.310867 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:42:09.310831    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:42:39.311111 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:42:39.311076    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:43:09.309626 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:43:09.309592    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:43:39.310337 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:43:39.310234    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:44:09.310660 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:44:09.310621    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:44:39.310132 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:44:39.310096    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:45:09.310314 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:45:09.310271    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:45:39.310058 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:45:39.310011    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:46:09.310116 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:46:09.310080    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:46:39.311458 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:46:39.311417    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:46:44.702842 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.702811    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"]
Oct 31 07:46:44.706022 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.706005    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.741200 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.741175    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d561dc40-ce33-4723-9420-8f76f5501610-tls-certs\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.741320 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.741209    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-model-cache\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.741320 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.741231    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-tmp-dir\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.741414 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.741316    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-home\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.741414 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.741405    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-dshm\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.741507 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.741457    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw62m\" (UniqueName: \"kubernetes.io/projected/d561dc40-ce33-4723-9420-8f76f5501610-kube-api-access-xw62m\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.807323 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.807287    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"openshift-service-ca.crt\""
Oct 31 07:46:44.807503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.807292    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"kube-root-ca.crt\""
Oct 31 07:46:44.807503 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.807400    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"default-dockercfg-9jmwn\""
Oct 31 07:46:44.807623 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.807533    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Oct 31 07:46:44.808799 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.808777    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"]
Oct 31 07:46:44.842817 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.842788    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d561dc40-ce33-4723-9420-8f76f5501610-tls-certs\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.842929 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.842820    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-model-cache\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.842929 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.842837    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-tmp-dir\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.842929 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.842865    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-home\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.842929 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.842881    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-dshm\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.843133 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.842931    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw62m\" (UniqueName: \"kubernetes.io/projected/d561dc40-ce33-4723-9420-8f76f5501610-kube-api-access-xw62m\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.843232 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.843210    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-model-cache\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.843298 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.843274    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-home\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.843353 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.843300    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-tmp-dir\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.845160 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.845133    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-dshm\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.845400 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.845383    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d561dc40-ce33-4723-9420-8f76f5501610-tls-certs\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:44.898753 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:44.898722    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw62m\" (UniqueName: \"kubernetes.io/projected/d561dc40-ce33-4723-9420-8f76f5501610-kube-api-access-xw62m\") pod \"scheduler-inline-config-test-kserve-5d95b5b968-fz4rv\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:45.017877 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:45.017819    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:45.383184 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:45.383142    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"]
Oct 31 07:46:45.385791 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:46:45.385762    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd561dc40_ce33_4723_9420_8f76f5501610.slice/crio-0b08c7246b4e05eaed618dd7f63b7850fe9c868c169ad87e23f8a728846c4059 WatchSource:0}: Error finding container 0b08c7246b4e05eaed618dd7f63b7850fe9c868c169ad87e23f8a728846c4059: Status 404 returned error can't find the container with id 0b08c7246b4e05eaed618dd7f63b7850fe9c868c169ad87e23f8a728846c4059
Oct 31 07:46:45.387511 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:45.387494    4532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 07:46:46.225885 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:46.225854    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" event={"ID":"d561dc40-ce33-4723-9420-8f76f5501610","Type":"ContainerStarted","Data":"0b08c7246b4e05eaed618dd7f63b7850fe9c868c169ad87e23f8a728846c4059"}
Oct 31 07:46:48.235345 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:48.235262    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" event={"ID":"d561dc40-ce33-4723-9420-8f76f5501610","Type":"ContainerStarted","Data":"d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d"}
Oct 31 07:46:48.348263 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:48.348215    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" podStartSLOduration=1.885349035 podStartE2EDuration="4.348201169s" totalImagesPullingTime="2.462852134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:46:48.34729726 +0000 UTC m=+1058.849451051" watchObservedRunningTime="2026-10-31 07:46:48.348201169 +0000 UTC m=+1058.850354958"
Oct 31 07:46:49.018039 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:49.018006    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:55.018397 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:55.018362    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:55.030842 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:55.030811    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:46:55.272410 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:46:55.272341    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:47:09.310479 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:47:09.310430    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:47:25.868075 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:25.868040    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"]
Oct 31 07:47:25.877960 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:47:25.877914    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d" cmd=["/bin/sleep","15"]
Oct 31 07:47:25.878093 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:25.877991    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 07:47:25.878093 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:47:25.878012    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" podUID="d561dc40-ce33-4723-9420-8f76f5501610" containerName="main" containerID="cri-o://d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d"
Oct 31 07:47:25.878093 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:25.878058    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" podUID="d561dc40-ce33-4723-9420-8f76f5501610" containerName="main" containerID="cri-o://d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d" gracePeriod=60
Oct 31 07:47:26.215513 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.215490    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:47:26.338969 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.338943    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-dshm\") pod \"d561dc40-ce33-4723-9420-8f76f5501610\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") "
Oct 31 07:47:26.339148 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.338983    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-tmp-dir\") pod \"d561dc40-ce33-4723-9420-8f76f5501610\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") "
Oct 31 07:47:26.339148 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339017    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d561dc40-ce33-4723-9420-8f76f5501610-tls-certs\" (UniqueName: \"kubernetes.io/secret/d561dc40-ce33-4723-9420-8f76f5501610-tls-certs\") pod \"d561dc40-ce33-4723-9420-8f76f5501610\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") "
Oct 31 07:47:26.339148 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339058    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-model-cache\") pod \"d561dc40-ce33-4723-9420-8f76f5501610\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") "
Oct 31 07:47:26.339148 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339086    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-home\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-home\") pod \"d561dc40-ce33-4723-9420-8f76f5501610\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") "
Oct 31 07:47:26.339378 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339157    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d561dc40-ce33-4723-9420-8f76f5501610-kube-api-access-xw62m\" (UniqueName: \"kubernetes.io/projected/d561dc40-ce33-4723-9420-8f76f5501610-kube-api-access-xw62m\") pod \"d561dc40-ce33-4723-9420-8f76f5501610\" (UID: \"d561dc40-ce33-4723-9420-8f76f5501610\") "
Oct 31 07:47:26.339378 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339275    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-tmp-dir" pod "d561dc40-ce33-4723-9420-8f76f5501610" (UID: "d561dc40-ce33-4723-9420-8f76f5501610"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:47:26.339378 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339358    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-model-cache" pod "d561dc40-ce33-4723-9420-8f76f5501610" (UID: "d561dc40-ce33-4723-9420-8f76f5501610"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:47:26.339559 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339378    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-home" pod "d561dc40-ce33-4723-9420-8f76f5501610" (UID: "d561dc40-ce33-4723-9420-8f76f5501610"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:47:26.339559 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.339447    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:47:26.341297 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.341270    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d561dc40-ce33-4723-9420-8f76f5501610-tls-certs" pod "d561dc40-ce33-4723-9420-8f76f5501610" (UID: "d561dc40-ce33-4723-9420-8f76f5501610"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:47:26.341297 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.341288    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-dshm" pod "d561dc40-ce33-4723-9420-8f76f5501610" (UID: "d561dc40-ce33-4723-9420-8f76f5501610"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:47:26.341443 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.341317    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d561dc40-ce33-4723-9420-8f76f5501610-kube-api-access-xw62m" pod "d561dc40-ce33-4723-9420-8f76f5501610" (UID: "d561dc40-ce33-4723-9420-8f76f5501610"). InnerVolumeSpecName "kube-api-access-xw62m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:47:26.370102 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.370079    4532 generic.go:356] "Generic (PLEG): container finished" podID="d561dc40-ce33-4723-9420-8f76f5501610" containerID="d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d" exitCode=0
Oct 31 07:47:26.370196 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.370121    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"
Oct 31 07:47:26.370196 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.370154    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" event={"ID":"d561dc40-ce33-4723-9420-8f76f5501610","Type":"ContainerDied","Data":"d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d"}
Oct 31 07:47:26.370265 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.370203    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv" event={"ID":"d561dc40-ce33-4723-9420-8f76f5501610","Type":"ContainerDied","Data":"0b08c7246b4e05eaed618dd7f63b7850fe9c868c169ad87e23f8a728846c4059"}
Oct 31 07:47:26.370265 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.370225    4532 scope.go:122] "RemoveContainer" containerID="d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d"
Oct 31 07:47:26.379053 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.379025    4532 scope.go:122] "RemoveContainer" containerID="d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d"
Oct 31 07:47:26.379315 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:47:26.379295    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d\": container with ID starting with d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d not found: ID does not exist" containerID="d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d"
Oct 31 07:47:26.379372 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.379324    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d"} err="failed to get container status \"d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d\": rpc error: code = NotFound desc = could not find container \"d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d\": container with ID starting with d3e95dad24ca264b4a0021764b28d162ac0dc75c2af422257cb8d31fa8bc2a7d not found: ID does not exist"
Oct 31 07:47:26.440506 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.440442    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:47:26.440506 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.440473    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:47:26.440506 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.440485    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw62m\" (UniqueName: \"kubernetes.io/projected/d561dc40-ce33-4723-9420-8f76f5501610-kube-api-access-xw62m\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:47:26.440506 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.440494    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d561dc40-ce33-4723-9420-8f76f5501610-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:47:26.440506 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.440503    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d561dc40-ce33-4723-9420-8f76f5501610-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:47:26.531894 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.531873    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"]
Oct 31 07:47:26.534298 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:26.534278    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-5d95b5b968-fz4rv"]
Oct 31 07:47:27.623092 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:47:27.623055    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d561dc40-ce33-4723-9420-8f76f5501610" path="/var/lib/kubelet/pods/d561dc40-ce33-4723-9420-8f76f5501610/volumes"
Oct 31 07:47:39.310456 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:47:39.310415    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:48:09.311712 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:48:09.311670    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:48:35.627645 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.627613    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"]
Oct 31 07:48:35.628177 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.628027    4532 cpu_manager.go:414] "containerMap: removing container" podUID="d561dc40-ce33-4723-9420-8f76f5501610" containerName="main"
Oct 31 07:48:35.628177 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.628044    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d561dc40-ce33-4723-9420-8f76f5501610" containerName="main"
Oct 31 07:48:35.628177 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.628164    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="d561dc40-ce33-4723-9420-8f76f5501610" containerName="main"
Oct 31 07:48:35.630867 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.630847    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.682711 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.682686    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 07:48:35.717917 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.717891    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718033 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.717924    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab696e69-e4bd-4141-bc1d-116c1490b720-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718033 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.717950    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718033 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.718004    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718184 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.718091    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718226 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.718194    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718262 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.718250    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718310 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.718268    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrp87\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-kube-api-access-hrp87\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.718310 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.718293    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.732036 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.732012    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-z29g8\""
Oct 31 07:48:35.732178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.732158    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 07:48:35.732245 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.732184    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 07:48:35.733510 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.733488    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"]
Oct 31 07:48:35.819622 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819593    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.819773 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819635    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.819773 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819651    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrp87\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-kube-api-access-hrp87\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.819773 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819676    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.819773 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819706    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.819773 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819736    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab696e69-e4bd-4141-bc1d-116c1490b720-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.819773 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819770    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.820077 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819793    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.820077 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.819865    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.820077 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.820051    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.820249 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.820126    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.820249 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.820189    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.820249 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.820245    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.820477 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.820446    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab696e69-e4bd-4141-bc1d-116c1490b720-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.821985 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.821964    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.822213 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.822194    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.875162 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.875137    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.875292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.875274    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrp87\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-kube-api-access-hrp87\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-tgnls\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:35.942931 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:35.942871    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:36.361241 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:36.361214    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"]
Oct 31 07:48:36.363764 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:48:36.363738    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab696e69_e4bd_4141_bc1d_116c1490b720.slice/crio-4bc9b08e3321a4980b11955637e069cb219fb9abc6e0d13eaa1cc14cd952325f WatchSource:0}: Error finding container 4bc9b08e3321a4980b11955637e069cb219fb9abc6e0d13eaa1cc14cd952325f: Status 404 returned error can't find the container with id 4bc9b08e3321a4980b11955637e069cb219fb9abc6e0d13eaa1cc14cd952325f
Oct 31 07:48:36.626395 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:36.626326    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" event={"ID":"ab696e69-e4bd-4141-bc1d-116c1490b720","Type":"ContainerStarted","Data":"4bc9b08e3321a4980b11955637e069cb219fb9abc6e0d13eaa1cc14cd952325f"}
Oct 31 07:48:39.313275 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:48:39.313162    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:48:39.952301 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:39.952262    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:48:39.952437 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:39.952336    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:48:39.952437 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:39.952377    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:48:40.645792 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:40.645757    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" event={"ID":"ab696e69-e4bd-4141-bc1d-116c1490b720","Type":"ContainerStarted","Data":"32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869"}
Oct 31 07:48:40.810819 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:40.810767    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" podStartSLOduration=2.224817063 podStartE2EDuration="5.810752981s" totalImagesPullingTime="3.585935918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:48:40.809311638 +0000 UTC m=+1171.311465428" watchObservedRunningTime="2026-10-31 07:48:40.810752981 +0000 UTC m=+1171.312906761"
Oct 31 07:48:40.943311 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:40.943227    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:41.945432 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:41.945386    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Oct 31 07:48:41.945846 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:41.945453    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Oct 31 07:48:42.943388 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:42.943356    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Oct 31 07:48:42.943569 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:42.943407    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Oct 31 07:48:43.943792 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:43.943757    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Oct 31 07:48:43.944268 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:43.943817    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Oct 31 07:48:44.943689 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:44.943661    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body=
Oct 31 07:48:44.943850 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:44.943711    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused"
Oct 31 07:48:45.947384 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:45.947349    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:45.947813 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:45.947697    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:45.948311 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:45.948294    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 07:48:50.747777 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.747745    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"]
Oct 31 07:48:50.751081 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.751063    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.855151 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.855126    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"]
Oct 31 07:48:50.867539 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867513    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09802de6-13c5-40c7-a219-42515a5195fb-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867664 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867556    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867664 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867596    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867664 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867629    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09802de6-13c5-40c7-a219-42515a5195fb-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867772 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867696    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfk26\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-kube-api-access-wfk26\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867772 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867727    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867772 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867754    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867878 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867775    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.867878 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.867835    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969055 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969031    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09802de6-13c5-40c7-a219-42515a5195fb-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969181 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969063    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969181 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969083    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969181 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969097    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09802de6-13c5-40c7-a219-42515a5195fb-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969181 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969154    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfk26\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-kube-api-access-wfk26\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969392 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969202    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969392 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969242    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969392 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969263    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969392 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969289    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969611 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969518    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969648 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969610    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969731 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969712    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969817 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969752    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.969901 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.969879    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09802de6-13c5-40c7-a219-42515a5195fb-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.971571 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.971545    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:50.971679 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:50.971641    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09802de6-13c5-40c7-a219-42515a5195fb-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:51.029138 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.029112    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:51.029569 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.029551    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfk26\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-kube-api-access-wfk26\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:51.062519 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.062500    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:51.479607 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.479586    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"]
Oct 31 07:48:51.481296 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:48:51.481267    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09802de6_13c5_40c7_a219_42515a5195fb.slice/crio-88bd96164d0b7380ba946dbb21d76392a30e06c3c34804d1ac74ad043936efbb WatchSource:0}: Error finding container 88bd96164d0b7380ba946dbb21d76392a30e06c3c34804d1ac74ad043936efbb: Status 404 returned error can't find the container with id 88bd96164d0b7380ba946dbb21d76392a30e06c3c34804d1ac74ad043936efbb
Oct 31 07:48:51.486350 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.486314    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:48:51.486488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.486392    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:48:51.486488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.486434    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:48:51.703705 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.703673    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" event={"ID":"09802de6-13c5-40c7-a219-42515a5195fb","Type":"ContainerStarted","Data":"6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805"}
Oct 31 07:48:51.703868 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:51.703722    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" event={"ID":"09802de6-13c5-40c7-a219-42515a5195fb","Type":"ContainerStarted","Data":"88bd96164d0b7380ba946dbb21d76392a30e06c3c34804d1ac74ad043936efbb"}
Oct 31 07:48:52.080489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:52.072847    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:52.080489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:52.080042    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Oct 31 07:48:52.080489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:52.080099    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Oct 31 07:48:53.065810 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:53.065778    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Oct 31 07:48:53.066077 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:53.066049    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Oct 31 07:48:54.072493 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:54.071815    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Oct 31 07:48:54.072493 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:54.071877    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Oct 31 07:48:55.080489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:55.073845    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Oct 31 07:48:55.080489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:55.073912    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Oct 31 07:48:56.078487 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:56.074830    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Oct 31 07:48:56.078487 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:56.074904    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Oct 31 07:48:57.073851 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:57.073812    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Oct 31 07:48:57.074451 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:57.074418    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Oct 31 07:48:58.066561 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:58.066535    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:58.066844 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:58.066821    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:58.067451 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:58.067431    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 07:48:58.232298 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:48:58.232254    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podStartSLOduration=8.232239667 podStartE2EDuration="8.232239667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:48:51.821234268 +0000 UTC m=+1182.323388062" watchObservedRunningTime="2026-10-31 07:48:58.232239667 +0000 UTC m=+1188.734393456"
Oct 31 07:49:09.310137 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:49:09.310101    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:49:39.310962 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:49:39.310927    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:50:09.310628 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:50:09.310587    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:50:39.311045 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:50:39.310996    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:50:51.678497 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.678245    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"]
Oct 31 07:50:51.684006 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.683539    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.726261 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.726232    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"]
Oct 31 07:50:51.729336 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.729317    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.761786 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.761765    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.761907 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.761797    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.761907 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.761827    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762026 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.761908    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762026 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.761955    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hmjd\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-kube-api-access-2hmjd\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762111 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762035    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.762111 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762067    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.762111 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762101    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762276 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762250    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.762582 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762557    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762710 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762596    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6kk9\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-kube-api-access-d6kk9\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.762710 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762616    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.762710 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762636    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762710 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762653    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762710 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762681    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.762710 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762700    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.763014 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762719    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.763014 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.762740    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.825578 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.825552    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"]
Oct 31 07:50:51.863734 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.863706    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.863866 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.863739    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.863866 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.863759    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.863866 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.863787    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.863866 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.863811    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.863866 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.863849    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6kk9\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-kube-api-access-d6kk9\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864128 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.863889    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864200 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864178    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864257 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864178    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864257 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864198    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864257 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864230    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864578 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864257    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864655 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864626    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864712 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864420    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864712 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864662    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864712 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864443    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864712 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864525    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864712 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864690    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864734    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864775    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864829    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.864955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864858    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864886    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864917    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hmjd\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-kube-api-access-2hmjd\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.864955 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.864928    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.865295 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.865280    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.865474 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.865442    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.866089 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.866061    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.867455 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.867431    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.867669 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.867624    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.867809 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.867785    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.868438 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.868074    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.883361 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.883316    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"]
Oct 31 07:50:51.921854 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.921826    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6kk9\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-kube-api-access-d6kk9\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.922166 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.922147    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.922498 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.922454    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hmjd\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-kube-api-access-2hmjd\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:51.927645 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.927626    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:51.996616 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:51.996569    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:52.041011 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.040976    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:52.414826 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.414640    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"]
Oct 31 07:50:52.420512 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:50:52.420479    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aca2a6b_5d08_4fc4_be07_24cf10901c4a.slice/crio-af7c1f3fc04750663a252a018c619b002e22a252bcb4e919fb9f3c303b83124e WatchSource:0}: Error finding container af7c1f3fc04750663a252a018c619b002e22a252bcb4e919fb9f3c303b83124e: Status 404 returned error can't find the container with id af7c1f3fc04750663a252a018c619b002e22a252bcb4e919fb9f3c303b83124e
Oct 31 07:50:52.422760 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.422716    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:50:52.422854 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.422833    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:50:52.422895 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.422882    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:50:52.470719 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.470695    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"]
Oct 31 07:50:52.473581 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:50:52.473554    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b0b3541_527a_4f06_ac6b_c34c9fb8981e.slice/crio-5211e0bc38ba3c55abd95c35321b99b5f430ec7bedc9b231900a5f3f0f03995a WatchSource:0}: Error finding container 5211e0bc38ba3c55abd95c35321b99b5f430ec7bedc9b231900a5f3f0f03995a: Status 404 returned error can't find the container with id 5211e0bc38ba3c55abd95c35321b99b5f430ec7bedc9b231900a5f3f0f03995a
Oct 31 07:50:52.476095 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.476065    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:50:52.476167 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.476128    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:50:52.476167 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:52.476157    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:50:53.226120 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:53.226083    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" event={"ID":"3b0b3541-527a-4f06-ac6b-c34c9fb8981e","Type":"ContainerStarted","Data":"9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710"}
Oct 31 07:50:53.226570 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:53.226144    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" event={"ID":"3b0b3541-527a-4f06-ac6b-c34c9fb8981e","Type":"ContainerStarted","Data":"5211e0bc38ba3c55abd95c35321b99b5f430ec7bedc9b231900a5f3f0f03995a"}
Oct 31 07:50:53.235747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:53.235714    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" event={"ID":"1aca2a6b-5d08-4fc4-be07-24cf10901c4a","Type":"ContainerStarted","Data":"e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f"}
Oct 31 07:50:53.235965 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:53.235943    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" event={"ID":"1aca2a6b-5d08-4fc4-be07-24cf10901c4a","Type":"ContainerStarted","Data":"af7c1f3fc04750663a252a018c619b002e22a252bcb4e919fb9f3c303b83124e"}
Oct 31 07:50:53.585395 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:53.585333    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podStartSLOduration=2.585312268 podStartE2EDuration="2.585312268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:50:53.584940807 +0000 UTC m=+1304.087094599" watchObservedRunningTime="2026-10-31 07:50:53.585312268 +0000 UTC m=+1304.087466060"
Oct 31 07:50:53.586000 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:53.585953    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podStartSLOduration=2.585942085 podStartE2EDuration="2.585942085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:50:53.469746644 +0000 UTC m=+1303.971900451" watchObservedRunningTime="2026-10-31 07:50:53.585942085 +0000 UTC m=+1304.088095875"
Oct 31 07:50:54.004542 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.003820    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:54.015489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.013591    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Oct 31 07:50:54.015489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.013660    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Oct 31 07:50:54.046484 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.044796    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:54.055292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.053567    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 31 07:50:54.055292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.053626    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 31 07:50:54.997878 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.997838    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Oct 31 07:50:54.998524 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:54.998492    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Oct 31 07:50:55.042477 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:55.042439    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 31 07:50:55.042662 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:55.042641    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 31 07:50:56.001495 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:55.998822    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Oct 31 07:50:56.001495 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:55.998922    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Oct 31 07:50:56.045941 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:56.045907    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 31 07:50:56.046225 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:56.046194    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 31 07:50:57.011058 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:57.011003    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Oct 31 07:50:57.011704 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:57.011648    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Oct 31 07:50:57.042101 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:57.042061    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 31 07:50:57.042212 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:57.042144    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 31 07:50:57.998488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:57.997816    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Oct 31 07:50:57.998488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:57.997877    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Oct 31 07:50:58.041982 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:58.041949    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 31 07:50:58.042331 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:58.041998    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 31 07:50:59.000584 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:59.000553    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:59.000841 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:59.000811    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:59.001393 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:59.001375    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 07:50:59.045203 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:59.045184    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:59.045567 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:59.045447    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:50:59.045970 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:50:59.045953    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 07:51:09.310160 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:51:09.310125    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:51:39.309986 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:51:39.309941    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:52:09.310263 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:52:09.310224    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:52:39.310892 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:52:39.310857    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:53:09.311339 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:53:09.311299    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:53:39.310115 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:53:39.310079    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:54:09.310753 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:54:09.310718    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:54:29.706490 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.688969    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"]
Oct 31 07:54:29.710940 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.710913    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:29.808491 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.797436    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"]
Oct 31 07:54:29.835523 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.835175    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Oct 31 07:54:29.840626 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.840594    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Oct 31 07:54:29.874564 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.874531    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-bchcn\""
Oct 31 07:54:29.896497 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.886855    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Oct 31 07:54:29.897783 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.897748    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9145515-b649-4548-8498-f50008797163-tls-certs\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:29.898057 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.898040    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-home\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:29.921837 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.921805    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-dshm\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:29.922163 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.922144    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-model-cache\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:29.922364 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.922348    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-tmp-dir\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:29.922628 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:29.922590    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8bxn\" (UniqueName: \"kubernetes.io/projected/d9145515-b649-4548-8498-f50008797163-kube-api-access-g8bxn\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.023585 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.023497    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9145515-b649-4548-8498-f50008797163-tls-certs\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.023836 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.023813    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-home\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.024003 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.023990    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-dshm\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.024401 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.024381    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-model-cache\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.024723 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.024710    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-tmp-dir\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.025106 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.025092    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8bxn\" (UniqueName: \"kubernetes.io/projected/d9145515-b649-4548-8498-f50008797163-kube-api-access-g8bxn\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.025286 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.025019    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-tmp-dir\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.025366 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.024254    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-home\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.025663 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.025644    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-model-cache\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.026614 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.026597    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-dshm\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.031407 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.031383    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9145515-b649-4548-8498-f50008797163-tls-certs\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.100487 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.099500    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8bxn\" (UniqueName: \"kubernetes.io/projected/d9145515-b649-4548-8498-f50008797163-kube-api-access-g8bxn\") pod \"fc-concurrency-test-kserve-5485c7876d-5wwmk\" (UID: \"d9145515-b649-4548-8498-f50008797163\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.163487 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.162491    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:30.574145 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:30.574115    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"]
Oct 31 07:54:30.576376 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:54:30.576345    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9145515_b649_4548_8498_f50008797163.slice/crio-b2b26ca3637f69d899280f727346f98afb9e5b700fdcaf797c79f8706c9a624f WatchSource:0}: Error finding container b2b26ca3637f69d899280f727346f98afb9e5b700fdcaf797c79f8706c9a624f: Status 404 returned error can't find the container with id b2b26ca3637f69d899280f727346f98afb9e5b700fdcaf797c79f8706c9a624f
Oct 31 07:54:31.319094 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:31.319051    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" event={"ID":"d9145515-b649-4548-8498-f50008797163","Type":"ContainerStarted","Data":"d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800"}
Oct 31 07:54:31.319496 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:31.319116    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" event={"ID":"d9145515-b649-4548-8498-f50008797163","Type":"ContainerStarted","Data":"b2b26ca3637f69d899280f727346f98afb9e5b700fdcaf797c79f8706c9a624f"}
Oct 31 07:54:31.432918 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:31.432868    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" podStartSLOduration=2.432854505 podStartE2EDuration="2.432854505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:54:31.430917452 +0000 UTC m=+1521.933071245" watchObservedRunningTime="2026-10-31 07:54:31.432854505 +0000 UTC m=+1521.935008294"
Oct 31 07:54:32.163485 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:32.163445    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:39.310754 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:54:39.310721    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:54:40.163650 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:40.163618    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:40.176010 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:40.175984    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:54:40.364459 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:54:40.364430    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:55:09.310574 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:55:09.310536    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:55:13.387251 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.387208    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"]
Oct 31 07:55:13.396882 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:55:13.396840    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800" cmd=["/bin/sleep","15"]
Oct 31 07:55:13.397036 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.396908    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 07:55:13.397036 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:55:13.396926    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" podUID="d9145515-b649-4548-8498-f50008797163" containerName="main" containerID="cri-o://d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800"
Oct 31 07:55:13.397036 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.396983    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" podUID="d9145515-b649-4548-8498-f50008797163" containerName="main" containerID="cri-o://d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800" gracePeriod=60
Oct 31 07:55:13.741783 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.741758    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:55:13.779823 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.779795    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-home\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-home\") pod \"d9145515-b649-4548-8498-f50008797163\" (UID: \"d9145515-b649-4548-8498-f50008797163\") "
Oct 31 07:55:13.779939 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.779836    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-model-cache\") pod \"d9145515-b649-4548-8498-f50008797163\" (UID: \"d9145515-b649-4548-8498-f50008797163\") "
Oct 31 07:55:13.779939 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.779911    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-dshm\") pod \"d9145515-b649-4548-8498-f50008797163\" (UID: \"d9145515-b649-4548-8498-f50008797163\") "
Oct 31 07:55:13.780051 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.779996    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9145515-b649-4548-8498-f50008797163-kube-api-access-g8bxn\" (UniqueName: \"kubernetes.io/projected/d9145515-b649-4548-8498-f50008797163-kube-api-access-g8bxn\") pod \"d9145515-b649-4548-8498-f50008797163\" (UID: \"d9145515-b649-4548-8498-f50008797163\") "
Oct 31 07:55:13.780051 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780025    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-home" pod "d9145515-b649-4548-8498-f50008797163" (UID: "d9145515-b649-4548-8498-f50008797163"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:55:13.780458 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780052    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-tmp-dir\") pod \"d9145515-b649-4548-8498-f50008797163\" (UID: \"d9145515-b649-4548-8498-f50008797163\") "
Oct 31 07:55:13.780458 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780098    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9145515-b649-4548-8498-f50008797163-tls-certs\" (UniqueName: \"kubernetes.io/secret/d9145515-b649-4548-8498-f50008797163-tls-certs\") pod \"d9145515-b649-4548-8498-f50008797163\" (UID: \"d9145515-b649-4548-8498-f50008797163\") "
Oct 31 07:55:13.780458 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780112    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-model-cache" pod "d9145515-b649-4548-8498-f50008797163" (UID: "d9145515-b649-4548-8498-f50008797163"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:55:13.780458 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780263    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-tmp-dir" pod "d9145515-b649-4548-8498-f50008797163" (UID: "d9145515-b649-4548-8498-f50008797163"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:55:13.780458 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780357    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:55:13.780458 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780372    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:55:13.780458 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.780385    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:55:13.782095 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.782063    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9145515-b649-4548-8498-f50008797163-tls-certs" pod "d9145515-b649-4548-8498-f50008797163" (UID: "d9145515-b649-4548-8498-f50008797163"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:55:13.782196 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.782114    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-dshm" pod "d9145515-b649-4548-8498-f50008797163" (UID: "d9145515-b649-4548-8498-f50008797163"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:55:13.782196 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.782144    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9145515-b649-4548-8498-f50008797163-kube-api-access-g8bxn" pod "d9145515-b649-4548-8498-f50008797163" (UID: "d9145515-b649-4548-8498-f50008797163"). InnerVolumeSpecName "kube-api-access-g8bxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:55:13.880804 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.880778    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9145515-b649-4548-8498-f50008797163-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:55:13.880804 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.880803    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8bxn\" (UniqueName: \"kubernetes.io/projected/d9145515-b649-4548-8498-f50008797163-kube-api-access-g8bxn\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:55:13.880959 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:13.880815    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9145515-b649-4548-8498-f50008797163-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:55:14.495018 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.494982    4532 generic.go:356] "Generic (PLEG): container finished" podID="d9145515-b649-4548-8498-f50008797163" containerID="d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800" exitCode=0
Oct 31 07:55:14.495488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.495036    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" event={"ID":"d9145515-b649-4548-8498-f50008797163","Type":"ContainerDied","Data":"d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800"}
Oct 31 07:55:14.495488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.495077    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk" event={"ID":"d9145515-b649-4548-8498-f50008797163","Type":"ContainerDied","Data":"b2b26ca3637f69d899280f727346f98afb9e5b700fdcaf797c79f8706c9a624f"}
Oct 31 07:55:14.495488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.495106    4532 scope.go:122] "RemoveContainer" containerID="d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800"
Oct 31 07:55:14.495488 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.495036    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"
Oct 31 07:55:14.504673 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.504656    4532 scope.go:122] "RemoveContainer" containerID="d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800"
Oct 31 07:55:14.504927 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:55:14.504910    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800\": container with ID starting with d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800 not found: ID does not exist" containerID="d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800"
Oct 31 07:55:14.504996 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.504935    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800"} err="failed to get container status \"d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800\": rpc error: code = NotFound desc = could not find container \"d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800\": container with ID starting with d8f5b68349c6865ff01f14e1d20005dd7e5cbbe2c978da769a3ed6856c28b800 not found: ID does not exist"
Oct 31 07:55:14.655961 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.655935    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"]
Oct 31 07:55:14.705156 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:14.705133    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-5485c7876d-5wwmk"]
Oct 31 07:55:15.624202 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:55:15.624170    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9145515-b649-4548-8498-f50008797163" path="/var/lib/kubelet/pods/d9145515-b649-4548-8498-f50008797163/volumes"
Oct 31 07:55:39.310750 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:55:39.310715    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:56:09.311295 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:56:09.311251    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:56:39.310810 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:56:39.310763    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:57:09.310132 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:57:09.310096    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:57:34.367843 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.367811    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"]
Oct 31 07:57:34.368266 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.368145    4532 cpu_manager.go:414] "containerMap: removing container" podUID="d9145515-b649-4548-8498-f50008797163" containerName="main"
Oct 31 07:57:34.368266 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.368155    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9145515-b649-4548-8498-f50008797163" containerName="main"
Oct 31 07:57:34.368266 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.368215    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9145515-b649-4548-8498-f50008797163" containerName="main"
Oct 31 07:57:34.371130 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.371109    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.422633 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.422592    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\""
Oct 31 07:57:34.422790 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.422637    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\""
Oct 31 07:57:34.422790 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.422636    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-gw-ca-root-cert\""
Oct 31 07:57:34.422790 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.422662    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"router-gateway-1-openshift-default-dockercfg-4zsg6\""
Oct 31 07:57:34.436225 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436196    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmngt\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-kube-api-access-qmngt\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436372 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436246    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-envoy\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436372 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436299    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-certs\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436522 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436378    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a656275-4e5f-4610-b341-4356729c6da1-istio-podinfo\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436522 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436396    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a656275-4e5f-4610-b341-4356729c6da1-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436634 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436539    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-socket\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436634 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436584    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-istio-token\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436634 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436619    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-credential-socket\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.436796 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.436640    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-data\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.473511 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.473486    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"]
Oct 31 07:57:34.537359 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537325    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-credential-socket\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537359 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537362    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-data\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537631 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537503    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmngt\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-kube-api-access-qmngt\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537631 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537549    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-envoy\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537631 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537584    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-certs\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537631 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537615    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a656275-4e5f-4610-b341-4356729c6da1-istio-podinfo\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537848 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537643    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a656275-4e5f-4610-b341-4356729c6da1-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537848 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537693    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-socket\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537848 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537729    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-data\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537848 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537733    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-istio-token\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.537848 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.537800    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-credential-socket\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.538118 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.538098    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-certs\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.538178 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.538141    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-socket\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.538517 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.538495    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a656275-4e5f-4610-b341-4356729c6da1-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.539978 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.539957    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-envoy\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.540147 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.540129    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a656275-4e5f-4610-b341-4356729c6da1-istio-podinfo\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.593130 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.593100    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-istio-token\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.593130 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.593114    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmngt\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-kube-api-access-qmngt\") pod \"router-gateway-1-openshift-default-58b56c49d5-ghkww\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") " pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:34.683931 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:34.683870    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:35.103643 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:35.103614    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"]
Oct 31 07:57:35.106438 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:57:35.106409    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a656275_4e5f_4610_b341_4356729c6da1.slice/crio-7b2578f7b380fed8387fb0d0cb6e10ac55882b17911a56fc6827bd236b52620a WatchSource:0}: Error finding container 7b2578f7b380fed8387fb0d0cb6e10ac55882b17911a56fc6827bd236b52620a: Status 404 returned error can't find the container with id 7b2578f7b380fed8387fb0d0cb6e10ac55882b17911a56fc6827bd236b52620a
Oct 31 07:57:35.108252 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:35.108220    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:57:35.108318 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:35.108288    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:57:35.108354 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:35.108324    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 07:57:36.067862 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:36.067818    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww" event={"ID":"1a656275-4e5f-4610-b341-4356729c6da1","Type":"ContainerStarted","Data":"7052e984f2fc6b9d3350d5c1150bd9cd44e57d992b32a58d090bbf25dcfd0fcb"}
Oct 31 07:57:36.068361 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:36.067886    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww" event={"ID":"1a656275-4e5f-4610-b341-4356729c6da1","Type":"ContainerStarted","Data":"7b2578f7b380fed8387fb0d0cb6e10ac55882b17911a56fc6827bd236b52620a"}
Oct 31 07:57:36.232166 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:36.232107    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww" podStartSLOduration=2.23209224 podStartE2EDuration="2.23209224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:57:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:57:36.231156696 +0000 UTC m=+1706.733310488" watchObservedRunningTime="2026-10-31 07:57:36.23209224 +0000 UTC m=+1706.734246109"
Oct 31 07:57:36.685026 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:36.684988    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:36.690119 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:36.690092    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:37.072287 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:37.072258    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:37.073315 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:37.073297    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:38.948631 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:38.948595    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"]
Oct 31 07:57:38.952044 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:38.952023    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.052997 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.052969    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Oct 31 07:57:39.053147 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.053009    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"default-dockercfg-sm4xp\""
Oct 31 07:57:39.054547 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.054531    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"]
Oct 31 07:57:39.081292 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.081259    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.081424 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.081331    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8smk\" (UniqueName: \"kubernetes.io/projected/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-kube-api-access-m8smk\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.081424 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.081406    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.081536 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.081434    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.081593 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.081544    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.081640 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.081619    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.182689 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.182649    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.182689 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.182682    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.182910 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.182717    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.182910 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.182741    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.182910 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.182797    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.182910 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.182884    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8smk\" (UniqueName: \"kubernetes.io/projected/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-kube-api-access-m8smk\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.183119 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.183093    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.183176 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.183108    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.183284 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.183258    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.185199 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.185167    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.185454 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.185433    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.238758 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.238706    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8smk\" (UniqueName: \"kubernetes.io/projected/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-kube-api-access-m8smk\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.264656 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.264626    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:39.327523 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:57:39.327451    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:57:39.643270 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:39.643247    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"]
Oct 31 07:57:39.645845 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:57:39.645821    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c92d6f9_8cba_4fc9_be57_f76ec0f9d350.slice/crio-852b0342e8c460d5e3fc38785e4e60b981c8a0cf1b33e34bf6a3cd0e249cf83b WatchSource:0}: Error finding container 852b0342e8c460d5e3fc38785e4e60b981c8a0cf1b33e34bf6a3cd0e249cf83b: Status 404 returned error can't find the container with id 852b0342e8c460d5e3fc38785e4e60b981c8a0cf1b33e34bf6a3cd0e249cf83b
Oct 31 07:57:40.087093 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:40.087060    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" event={"ID":"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350","Type":"ContainerStarted","Data":"f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829"}
Oct 31 07:57:40.087497 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:40.087115    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" event={"ID":"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350","Type":"ContainerStarted","Data":"852b0342e8c460d5e3fc38785e4e60b981c8a0cf1b33e34bf6a3cd0e249cf83b"}
Oct 31 07:57:40.200426 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:40.200379    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" podStartSLOduration=2.200367893 podStartE2EDuration="2.200367893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:57:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:57:40.199213068 +0000 UTC m=+1710.701366859" watchObservedRunningTime="2026-10-31 07:57:40.200367893 +0000 UTC m=+1710.702521679"
Oct 31 07:57:40.265536 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:40.265511    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:46.491904 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.491865    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"]
Oct 31 07:57:46.501078 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:57:46.501040    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829" cmd=["/bin/sleep","15"]
Oct 31 07:57:46.501225 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.501094    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 07:57:46.501225 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:57:46.501108    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" podUID="8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" containerName="main" containerID="cri-o://f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829"
Oct 31 07:57:46.501225 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.501153    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" podUID="8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" containerName="main" containerID="cri-o://f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829" gracePeriod=60
Oct 31 07:57:46.541249 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.541222    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"]
Oct 31 07:57:46.541416 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.541397    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww" podUID="1a656275-4e5f-4610-b341-4356729c6da1" containerName="istio-proxy" containerID="cri-o://7052e984f2fc6b9d3350d5c1150bd9cd44e57d992b32a58d090bbf25dcfd0fcb" gracePeriod=30
Oct 31 07:57:46.751788 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.751731    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:46.847111 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847078    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-dshm\") pod \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") "
Oct 31 07:57:46.847219 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847128    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tls-certs\") pod \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") "
Oct 31 07:57:46.847219 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847156    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tmp-dir\") pod \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") "
Oct 31 07:57:46.847299 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847225    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-home\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-home\") pod \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") "
Oct 31 07:57:46.847444 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847425    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-home" pod "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" (UID: "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:46.847535 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847436    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tmp-dir" pod "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" (UID: "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:46.847535 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847430    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-model-cache\") pod \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") "
Oct 31 07:57:46.847535 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847524    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-kube-api-access-m8smk\" (UniqueName: \"kubernetes.io/projected/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-kube-api-access-m8smk\") pod \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\" (UID: \"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350\") "
Oct 31 07:57:46.847682 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847597    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-model-cache" pod "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" (UID: "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:46.847785 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847768    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:46.847843 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847786    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:46.847843 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.847796    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:46.849312 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.849287    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-dshm" pod "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" (UID: "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:46.849429 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.849372    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tls-certs" pod "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" (UID: "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:57:46.849429 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.849403    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-kube-api-access-m8smk" pod "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" (UID: "8c92d6f9-8cba-4fc9-be57-f76ec0f9d350"). InnerVolumeSpecName "kube-api-access-m8smk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:57:46.948382 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.948355    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8smk\" (UniqueName: \"kubernetes.io/projected/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-kube-api-access-m8smk\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:46.948382 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.948382    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:46.948579 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:46.948394    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:47.116601 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.116568    4532 generic.go:356] "Generic (PLEG): container finished" podID="8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" containerID="f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829" exitCode=0
Oct 31 07:57:47.116770 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.116616    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"
Oct 31 07:57:47.116770 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.116656    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" event={"ID":"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350","Type":"ContainerDied","Data":"f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829"}
Oct 31 07:57:47.116770 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.116714    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9" event={"ID":"8c92d6f9-8cba-4fc9-be57-f76ec0f9d350","Type":"ContainerDied","Data":"852b0342e8c460d5e3fc38785e4e60b981c8a0cf1b33e34bf6a3cd0e249cf83b"}
Oct 31 07:57:47.116770 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.116737    4532 scope.go:122] "RemoveContainer" containerID="f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829"
Oct 31 07:57:47.125841 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.125825    4532 scope.go:122] "RemoveContainer" containerID="f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829"
Oct 31 07:57:47.126101 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:57:47.126083    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829\": container with ID starting with f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829 not found: ID does not exist" containerID="f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829"
Oct 31 07:57:47.126174 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.126113    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829"} err="failed to get container status \"f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829\": rpc error: code = NotFound desc = could not find container \"f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829\": container with ID starting with f665a7dc8da725de665e25490b40e05a8fb33ead29337ff83c047c17c03c9829 not found: ID does not exist"
Oct 31 07:57:47.277711 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.277682    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"]
Oct 31 07:57:47.282948 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.282925    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-7l7b9"]
Oct 31 07:57:47.624106 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:47.624075    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" path="/var/lib/kubelet/pods/8c92d6f9-8cba-4fc9-be57-f76ec0f9d350/volumes"
Oct 31 07:57:52.080489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.079908    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww" podUID="1a656275-4e5f-4610-b341-4356729c6da1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Oct 31 07:57:52.483817 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.483682    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww" event={"ID":"1a656275-4e5f-4610-b341-4356729c6da1","Type":"ContainerDied","Data":"7052e984f2fc6b9d3350d5c1150bd9cd44e57d992b32a58d090bbf25dcfd0fcb"}
Oct 31 07:57:52.484037 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.484008    4532 generic.go:356] "Generic (PLEG): container finished" podID="1a656275-4e5f-4610-b341-4356729c6da1" containerID="7052e984f2fc6b9d3350d5c1150bd9cd44e57d992b32a58d090bbf25dcfd0fcb" exitCode=0
Oct 31 07:57:52.604599 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.604272    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.684782    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-data\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.684856    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a656275-4e5f-4610-b341-4356729c6da1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a656275-4e5f-4610-b341-4356729c6da1-istio-podinfo\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.684940    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-credential-socket\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.684987    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-istio-token\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-istio-token\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.685034    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a656275-4e5f-4610-b341-4356729c6da1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a656275-4e5f-4610-b341-4356729c6da1-istiod-ca-cert\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.685074    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-envoy\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.685108    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-kube-api-access-qmngt\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-kube-api-access-qmngt\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.685155    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-certs\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.685738 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.685188    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-socket\") pod \"1a656275-4e5f-4610-b341-4356729c6da1\" (UID: \"1a656275-4e5f-4610-b341-4356729c6da1\") "
Oct 31 07:57:52.686828 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.686420    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-socket" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:52.692754 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.691774    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a656275-4e5f-4610-b341-4356729c6da1-istiod-ca-cert" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 07:57:52.692754 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.692319    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-credential-socket" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:52.693155 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.692978    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-certs" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:52.693769 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.693294    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-data" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:52.711855 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.707140    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-envoy" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:57:52.711855 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.710621    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-istio-token" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:57:52.720644 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.720583    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a656275-4e5f-4610-b341-4356729c6da1-istio-podinfo" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 07:57:52.723577 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.723541    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-kube-api-access-qmngt" pod "1a656275-4e5f-4610-b341-4356729c6da1" (UID: "1a656275-4e5f-4610-b341-4356729c6da1"). InnerVolumeSpecName "kube-api-access-qmngt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799269    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a656275-4e5f-4610-b341-4356729c6da1-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799305    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799322    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799337    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a656275-4e5f-4610-b341-4356729c6da1-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799352    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799366    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmngt\" (UniqueName: \"kubernetes.io/projected/1a656275-4e5f-4610-b341-4356729c6da1-kube-api-access-qmngt\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799380    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799394    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:52.801489 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:52.799410    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a656275-4e5f-4610-b341-4356729c6da1-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:57:53.504549 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:53.500192    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww" event={"ID":"1a656275-4e5f-4610-b341-4356729c6da1","Type":"ContainerDied","Data":"7b2578f7b380fed8387fb0d0cb6e10ac55882b17911a56fc6827bd236b52620a"}
Oct 31 07:57:53.504549 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:53.500268    4532 scope.go:122] "RemoveContainer" containerID="7052e984f2fc6b9d3350d5c1150bd9cd44e57d992b32a58d090bbf25dcfd0fcb"
Oct 31 07:57:53.504549 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:53.500411    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"
Oct 31 07:57:53.730697 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:53.726117    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"]
Oct 31 07:57:53.788747 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:53.786009    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/router-gateway-1-openshift-default-58b56c49d5-ghkww"]
Oct 31 07:57:55.639931 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:57:55.639896    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a656275-4e5f-4610-b341-4356729c6da1" path="/var/lib/kubelet/pods/1a656275-4e5f-4610-b341-4356729c6da1/volumes"
Oct 31 07:58:03.935513 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.935459    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"]
Oct 31 07:58:03.936078 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.935928    4532 cpu_manager.go:414] "containerMap: removing container" podUID="1a656275-4e5f-4610-b341-4356729c6da1" containerName="istio-proxy"
Oct 31 07:58:03.936078 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.935948    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a656275-4e5f-4610-b341-4356729c6da1" containerName="istio-proxy"
Oct 31 07:58:03.936078 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.935996    4532 cpu_manager.go:414] "containerMap: removing container" podUID="8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" containerName="main"
Oct 31 07:58:03.936078 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.936004    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" containerName="main"
Oct 31 07:58:03.936293 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.936120    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a656275-4e5f-4610-b341-4356729c6da1" containerName="istio-proxy"
Oct 31 07:58:03.936293 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.936139    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c92d6f9-8cba-4fc9-be57-f76ec0f9d350" containerName="main"
Oct 31 07:58:03.939210 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.939188    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:03.989120 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.989092    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:03.989271 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.989138    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-home\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:03.989271 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.989176    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-dshm\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:03.989361 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.989267    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-model-cache\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:03.989361 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.989320    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxtbh\" (UniqueName: \"kubernetes.io/projected/99204538-b838-4088-b273-bdedfb4a0ee2-kube-api-access-rxtbh\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:03.989361 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:03.989341    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99204538-b838-4088-b273-bdedfb4a0ee2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.043813 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.043781    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Oct 31 07:58:04.043813 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.043781    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Oct 31 07:58:04.044031 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.043839    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Oct 31 07:58:04.044031 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.043859    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-h92jf\""
Oct 31 07:58:04.045431 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.045405    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"]
Oct 31 07:58:04.089869 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.089842    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.089973 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.089883    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-home\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.089973 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.089910    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-dshm\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.089973 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.089931    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-model-cache\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.089973 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.089960    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxtbh\" (UniqueName: \"kubernetes.io/projected/99204538-b838-4088-b273-bdedfb4a0ee2-kube-api-access-rxtbh\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.090137 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.089985    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99204538-b838-4088-b273-bdedfb4a0ee2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.090282 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.090253    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.090398 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.090283    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-home\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.090444 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.090396    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-model-cache\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.092124 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.092104    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-dshm\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.092531 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.092513    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99204538-b838-4088-b273-bdedfb4a0ee2-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.146029 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.146000    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxtbh\" (UniqueName: \"kubernetes.io/projected/99204538-b838-4088-b273-bdedfb4a0ee2-kube-api-access-rxtbh\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.253909 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.253827    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:04.620110 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:04.620079    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"]
Oct 31 07:58:04.623537 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:58:04.623507    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99204538_b838_4088_b273_bdedfb4a0ee2.slice/crio-0eb8dfb736eed930d760207f0cd4482bdb52c2caf1bebddbe2c5d735077fcff8 WatchSource:0}: Error finding container 0eb8dfb736eed930d760207f0cd4482bdb52c2caf1bebddbe2c5d735077fcff8: Status 404 returned error can't find the container with id 0eb8dfb736eed930d760207f0cd4482bdb52c2caf1bebddbe2c5d735077fcff8
Oct 31 07:58:05.569361 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:05.569320    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" event={"ID":"99204538-b838-4088-b273-bdedfb4a0ee2","Type":"ContainerStarted","Data":"d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b"}
Oct 31 07:58:05.569856 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:05.569384    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" event={"ID":"99204538-b838-4088-b273-bdedfb4a0ee2","Type":"ContainerStarted","Data":"0eb8dfb736eed930d760207f0cd4482bdb52c2caf1bebddbe2c5d735077fcff8"}
Oct 31 07:58:05.681196 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:05.681139    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" podStartSLOduration=2.681121253 podStartE2EDuration="2.681121253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:58:05.680871586 +0000 UTC m=+1736.183025379" watchObservedRunningTime="2026-10-31 07:58:05.681121253 +0000 UTC m=+1736.183275045"
Oct 31 07:58:06.254636 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:06.254605    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:09.311992 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:58:09.311953    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:58:14.254408 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:14.254376    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:14.267209 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:14.267179    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:14.618443 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:14.618409    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:39.309001 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:58:39.308964    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:58:43.525042 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.525005    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"]
Oct 31 07:58:43.535301 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:58:43.535264    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b" cmd=["/bin/sleep","15"]
Oct 31 07:58:43.535441 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.535320    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 07:58:43.535441 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:58:43.535336    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" podUID="99204538-b838-4088-b273-bdedfb4a0ee2" containerName="main" containerID="cri-o://d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b"
Oct 31 07:58:43.535441 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.535374    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" podUID="99204538-b838-4088-b273-bdedfb4a0ee2" containerName="main" containerID="cri-o://d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b" gracePeriod=60
Oct 31 07:58:43.607822 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.607788    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" podUID="99204538-b838-4088-b273-bdedfb4a0ee2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/health\": dial tcp 10.133.0.31:8000: connect: connection refused"
Oct 31 07:58:43.871004 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.870976    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:43.928624 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928599    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-model-cache\") pod \"99204538-b838-4088-b273-bdedfb4a0ee2\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") "
Oct 31 07:58:43.928748 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928632    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99204538-b838-4088-b273-bdedfb4a0ee2-kube-api-access-rxtbh\" (UniqueName: \"kubernetes.io/projected/99204538-b838-4088-b273-bdedfb4a0ee2-kube-api-access-rxtbh\") pod \"99204538-b838-4088-b273-bdedfb4a0ee2\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") "
Oct 31 07:58:43.928748 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928661    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-tmp-dir\") pod \"99204538-b838-4088-b273-bdedfb4a0ee2\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") "
Oct 31 07:58:43.928748 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928696    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99204538-b838-4088-b273-bdedfb4a0ee2-tls-certs\" (UniqueName: \"kubernetes.io/secret/99204538-b838-4088-b273-bdedfb4a0ee2-tls-certs\") pod \"99204538-b838-4088-b273-bdedfb4a0ee2\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") "
Oct 31 07:58:43.928900 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928745    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-home\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-home\") pod \"99204538-b838-4088-b273-bdedfb4a0ee2\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") "
Oct 31 07:58:43.928900 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928772    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-dshm\") pod \"99204538-b838-4088-b273-bdedfb4a0ee2\" (UID: \"99204538-b838-4088-b273-bdedfb4a0ee2\") "
Oct 31 07:58:43.928900 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928886    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-tmp-dir" pod "99204538-b838-4088-b273-bdedfb4a0ee2" (UID: "99204538-b838-4088-b273-bdedfb4a0ee2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:58:43.929054 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.928910    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-model-cache" pod "99204538-b838-4088-b273-bdedfb4a0ee2" (UID: "99204538-b838-4088-b273-bdedfb4a0ee2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:58:43.929054 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.929022    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-home" pod "99204538-b838-4088-b273-bdedfb4a0ee2" (UID: "99204538-b838-4088-b273-bdedfb4a0ee2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:58:43.929129 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.929094    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:58:43.929129 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.929123    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:58:43.930829 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.930807    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-dshm" pod "99204538-b838-4088-b273-bdedfb4a0ee2" (UID: "99204538-b838-4088-b273-bdedfb4a0ee2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 07:58:43.930931 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.930861    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99204538-b838-4088-b273-bdedfb4a0ee2-tls-certs" pod "99204538-b838-4088-b273-bdedfb4a0ee2" (UID: "99204538-b838-4088-b273-bdedfb4a0ee2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 07:58:43.930931 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:43.930895    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99204538-b838-4088-b273-bdedfb4a0ee2-kube-api-access-rxtbh" pod "99204538-b838-4088-b273-bdedfb4a0ee2" (UID: "99204538-b838-4088-b273-bdedfb4a0ee2"). InnerVolumeSpecName "kube-api-access-rxtbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 07:58:44.029835 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.029802    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxtbh\" (UniqueName: \"kubernetes.io/projected/99204538-b838-4088-b273-bdedfb4a0ee2-kube-api-access-rxtbh\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:58:44.029835 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.029832    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99204538-b838-4088-b273-bdedfb4a0ee2-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:58:44.030025 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.029842    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:58:44.030025 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.029850    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99204538-b838-4088-b273-bdedfb4a0ee2-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 07:58:44.724802 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.724770    4532 generic.go:356] "Generic (PLEG): container finished" podID="99204538-b838-4088-b273-bdedfb4a0ee2" containerID="d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b" exitCode=0
Oct 31 07:58:44.725296 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.724811    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"
Oct 31 07:58:44.725296 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.724862    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" event={"ID":"99204538-b838-4088-b273-bdedfb4a0ee2","Type":"ContainerDied","Data":"d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b"}
Oct 31 07:58:44.725296 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.724904    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq" event={"ID":"99204538-b838-4088-b273-bdedfb4a0ee2","Type":"ContainerDied","Data":"0eb8dfb736eed930d760207f0cd4482bdb52c2caf1bebddbe2c5d735077fcff8"}
Oct 31 07:58:44.725296 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.724925    4532 scope.go:122] "RemoveContainer" containerID="d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b"
Oct 31 07:58:44.734234 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.734211    4532 scope.go:122] "RemoveContainer" containerID="d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b"
Oct 31 07:58:44.734530 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:58:44.734507    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b\": container with ID starting with d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b not found: ID does not exist" containerID="d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b"
Oct 31 07:58:44.734593 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.734536    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b"} err="failed to get container status \"d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b\": rpc error: code = NotFound desc = could not find container \"d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b\": container with ID starting with d48afd4dcfa6eb31c21e2a61e4ffc04c7befde9db9082d935e516ec3e1982c3b not found: ID does not exist"
Oct 31 07:58:44.884885 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.884859    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"]
Oct 31 07:58:44.934123 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:44.934098    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-z5mzq"]
Oct 31 07:58:45.625068 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:58:45.625031    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99204538-b838-4088-b273-bdedfb4a0ee2" path="/var/lib/kubelet/pods/99204538-b838-4088-b273-bdedfb4a0ee2/volumes"
Oct 31 07:59:09.310536 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:59:09.310499    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:59:39.310366 ip-10-0-133-50 kubenswrapper[4532]: E1031 07:59:39.310330    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 07:59:39.549861 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.549832    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"]
Oct 31 07:59:39.550153 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.550141    4532 cpu_manager.go:414] "containerMap: removing container" podUID="99204538-b838-4088-b273-bdedfb4a0ee2" containerName="main"
Oct 31 07:59:39.550208 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.550154    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99204538-b838-4088-b273-bdedfb4a0ee2" containerName="main"
Oct 31 07:59:39.550277 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.550264    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="99204538-b838-4088-b273-bdedfb4a0ee2" containerName="main"
Oct 31 07:59:39.553170 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.553154    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.654436 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.654351    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"kube-root-ca.crt\""
Oct 31 07:59:39.654436 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.654365    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"default-dockercfg-zfb55\""
Oct 31 07:59:39.654436 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.654392    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Oct 31 07:59:39.654436 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.654393    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"openshift-service-ca.crt\""
Oct 31 07:59:39.655789 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.655771    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"]
Oct 31 07:59:39.750046 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.750009    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tls-certs\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.750046 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.750060    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmmwt\" (UniqueName: \"kubernetes.io/projected/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-kube-api-access-lmmwt\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.750230 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.750177    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-dshm\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.750230 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.750201    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tmp-dir\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.750331 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.750309    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-model-cache\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.750391 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.750371    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-home\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851088 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851044    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tls-certs\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851088 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851093    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmmwt\" (UniqueName: \"kubernetes.io/projected/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-kube-api-access-lmmwt\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851330 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851139    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-dshm\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851330 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851155    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tmp-dir\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851330 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851191    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-model-cache\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851330 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851222    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-home\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851660 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851635    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-model-cache\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851660 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851652    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tmp-dir\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.851751 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.851703    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-home\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.853427 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.853401    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-dshm\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.853564 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.853513    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tls-certs\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:39.906793 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:39.906728    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmmwt\" (UniqueName: \"kubernetes.io/projected/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-kube-api-access-lmmwt\") pod \"scheduler-custom-template-test-kserve-6667fc56b8-2c6lq\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:40.165864 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:40.165777    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:40.534294 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:40.534263    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"]
Oct 31 07:59:40.537136 ip-10-0-133-50 kubenswrapper[4532]: W1031 07:59:40.537110    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36e2cd5_aaeb_45dd_b35e_35d449fcb4eb.slice/crio-5c9a38571e42856a5ea50306d035879ddec08e72526cb33e2cbfa7ab69824b27 WatchSource:0}: Error finding container 5c9a38571e42856a5ea50306d035879ddec08e72526cb33e2cbfa7ab69824b27: Status 404 returned error can't find the container with id 5c9a38571e42856a5ea50306d035879ddec08e72526cb33e2cbfa7ab69824b27
Oct 31 07:59:40.937839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:40.937750    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" event={"ID":"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb","Type":"ContainerStarted","Data":"a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2"}
Oct 31 07:59:40.937839 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:40.937817    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" event={"ID":"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb","Type":"ContainerStarted","Data":"5c9a38571e42856a5ea50306d035879ddec08e72526cb33e2cbfa7ab69824b27"}
Oct 31 07:59:41.050344 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:41.050293    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" podStartSLOduration=2.050279145 podStartE2EDuration="2.050279145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:59:41.049035302 +0000 UTC m=+1831.551189092" watchObservedRunningTime="2026-10-31 07:59:41.050279145 +0000 UTC m=+1831.552432935"
Oct 31 07:59:41.166538 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:41.166511    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:50.166782 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:50.166746    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:50.184898 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:50.184873    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 07:59:50.988496 ip-10-0-133-50 kubenswrapper[4532]: I1031 07:59:50.988405    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 08:00:09.310915 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:00:09.310882    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:00:18.639445 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:18.639406    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"]
Oct 31 08:00:18.653121 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:00:18.653087    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2" cmd=["/bin/sleep","15"]
Oct 31 08:00:18.653279 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:18.653141    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:00:18.653279 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:00:18.653154    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" podUID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" containerName="main" containerID="cri-o://a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2"
Oct 31 08:00:18.653279 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:18.653183    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" podUID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" containerName="main" containerID="cri-o://a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2" gracePeriod=60
Oct 31 08:00:18.999976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:18.999952    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 08:00:19.066873 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.066853    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-kube-api-access-lmmwt\" (UniqueName: \"kubernetes.io/projected/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-kube-api-access-lmmwt\") pod \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") "
Oct 31 08:00:19.066983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.066894    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tls-certs\") pod \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") "
Oct 31 08:00:19.066983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.066922    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-dshm\") pod \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") "
Oct 31 08:00:19.066983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.066958    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-model-cache\") pod \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") "
Oct 31 08:00:19.067135 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.066986    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tmp-dir\") pod \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") "
Oct 31 08:00:19.067135 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.067010    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-home\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-home\") pod \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\" (UID: \"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb\") "
Oct 31 08:00:19.067310 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.067277    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-model-cache" pod "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" (UID: "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:00:19.067310 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.067295    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tmp-dir" pod "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" (UID: "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:00:19.067496 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.067400    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-home" pod "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" (UID: "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:00:19.067496 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.067418    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:00:19.067496 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.067435    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:00:19.069147 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.069119    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tls-certs" pod "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" (UID: "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:00:19.069245 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.069169    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-dshm" pod "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" (UID: "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:00:19.069245 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.069203    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-kube-api-access-lmmwt" pod "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" (UID: "f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb"). InnerVolumeSpecName "kube-api-access-lmmwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:00:19.102158 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.102132    4532 generic.go:356] "Generic (PLEG): container finished" podID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" containerID="a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2" exitCode=0
Oct 31 08:00:19.102260 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.102167    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" event={"ID":"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb","Type":"ContainerDied","Data":"a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2"}
Oct 31 08:00:19.102260 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.102184    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"
Oct 31 08:00:19.102260 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.102208    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" event={"ID":"f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb","Type":"ContainerDied","Data":"5c9a38571e42856a5ea50306d035879ddec08e72526cb33e2cbfa7ab69824b27"}
Oct 31 08:00:19.102260 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.102234    4532 scope.go:122] "RemoveContainer" containerID="a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2"
Oct 31 08:00:19.111218 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.111198    4532 scope.go:122] "RemoveContainer" containerID="a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2"
Oct 31 08:00:19.111504 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:00:19.111477    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2\": container with ID starting with a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2 not found: ID does not exist" containerID="a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2"
Oct 31 08:00:19.111621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.111510    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2"} err="failed to get container status \"a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2\": rpc error: code = NotFound desc = could not find container \"a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2\": container with ID starting with a74ca00cf9e70bce04e235b26b5f1b4166beb208bb52b5788f1854168639dbc2 not found: ID does not exist"
Oct 31 08:00:19.168677 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.168652    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:00:19.168677 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.168675    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:00:19.168812 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.168685    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmmwt\" (UniqueName: \"kubernetes.io/projected/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-kube-api-access-lmmwt\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:00:19.168812 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.168693    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:00:19.220647 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.220593    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"]
Oct 31 08:00:19.224685 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.224666    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq"]
Oct 31 08:00:19.623758 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.623727    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" path="/var/lib/kubelet/pods/f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb/volumes"
Oct 31 08:00:19.978582 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:00:19.978506    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-6667fc56b8-2c6lq" podUID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 08:00:39.310557 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:00:39.310521    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:01:09.311118 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:01:09.311057    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:01:12.471708 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.471679    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"]
Oct 31 08:01:12.472078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.471994    4532 cpu_manager.go:414] "containerMap: removing container" podUID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" containerName="main"
Oct 31 08:01:12.472078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.472004    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" containerName="main"
Oct 31 08:01:12.472148 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.472088    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="f36e2cd5-aaeb-45dd-b35e-35d449fcb4eb" containerName="main"
Oct 31 08:01:12.475990 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.475971    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.531299 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.531269    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"]
Oct 31 08:01:12.554820 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.554790    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.554976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.554836    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flx4f\" (UniqueName: \"kubernetes.io/projected/61a3a6a7-e308-4fbe-8d2d-13038a63790d-kube-api-access-flx4f\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.554976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.554856    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.554976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.554931    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.555173 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.555002    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.555173 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.555023    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.578524 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.578498    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-d8tqr\""
Oct 31 08:01:12.578678 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.578524    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Oct 31 08:01:12.578678 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.578526    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Oct 31 08:01:12.578678 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.578549    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Oct 31 08:01:12.656259 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656236    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656377 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656268    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656377 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656286    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656377 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656354    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656526 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656393    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flx4f\" (UniqueName: \"kubernetes.io/projected/61a3a6a7-e308-4fbe-8d2d-13038a63790d-kube-api-access-flx4f\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656526 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656419    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656685 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656665    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656744 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656693    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.656788 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.656749    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.658528 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.658506    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.658719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.658701    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.712164 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.712134    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flx4f\" (UniqueName: \"kubernetes.io/projected/61a3a6a7-e308-4fbe-8d2d-13038a63790d-kube-api-access-flx4f\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:12.787759 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:12.787728    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:13.155039 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:13.154963    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"]
Oct 31 08:01:13.158018 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:01:13.157991    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a3a6a7_e308_4fbe_8d2d_13038a63790d.slice/crio-0a0ba6758fa902d9648bd46c9cc32b43dbf29bcce46537202278688ee176f21c WatchSource:0}: Error finding container 0a0ba6758fa902d9648bd46c9cc32b43dbf29bcce46537202278688ee176f21c: Status 404 returned error can't find the container with id 0a0ba6758fa902d9648bd46c9cc32b43dbf29bcce46537202278688ee176f21c
Oct 31 08:01:13.314589 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:13.314556    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" event={"ID":"61a3a6a7-e308-4fbe-8d2d-13038a63790d","Type":"ContainerStarted","Data":"c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45"}
Oct 31 08:01:13.314743 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:13.314605    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" event={"ID":"61a3a6a7-e308-4fbe-8d2d-13038a63790d","Type":"ContainerStarted","Data":"0a0ba6758fa902d9648bd46c9cc32b43dbf29bcce46537202278688ee176f21c"}
Oct 31 08:01:13.428104 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:13.427992    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" podStartSLOduration=1.427975081 podStartE2EDuration="1.427975081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:01:13.426874752 +0000 UTC m=+1923.929028542" watchObservedRunningTime="2026-10-31 08:01:13.427975081 +0000 UTC m=+1923.930128872"
Oct 31 08:01:13.788086 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:13.788060    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:22.788514 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:22.788422    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:22.817394 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:22.817352    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:23.457714 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:23.457681    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:39.311448 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:01:39.311408    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:01:55.415216 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.415179    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"]
Oct 31 08:01:55.424429 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:01:55.424383    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45" cmd=["/bin/sleep","15"]
Oct 31 08:01:55.424585 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.424455    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:01:55.424585 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:01:55.424502    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" podUID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" containerName="main" containerID="cri-o://c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45"
Oct 31 08:01:55.424585 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.424553    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" podUID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" containerName="main" containerID="cri-o://c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45" gracePeriod=60
Oct 31 08:01:55.447619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.447589    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" podUID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.33:8000/health\": dial tcp 10.133.0.33:8000: connect: connection refused"
Oct 31 08:01:55.765929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.765900    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:55.872726 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.872697    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61a3a6a7-e308-4fbe-8d2d-13038a63790d-kube-api-access-flx4f\" (UniqueName: \"kubernetes.io/projected/61a3a6a7-e308-4fbe-8d2d-13038a63790d-kube-api-access-flx4f\") pod \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") "
Oct 31 08:01:55.872853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.872740    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tmp-dir\") pod \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") "
Oct 31 08:01:55.872853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.872765    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-dshm\") pod \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") "
Oct 31 08:01:55.872853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.872810    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tls-certs\" (UniqueName: \"kubernetes.io/secret/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tls-certs\") pod \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") "
Oct 31 08:01:55.873009 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.872868    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-home\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-home\") pod \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") "
Oct 31 08:01:55.873009 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.872899    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-model-cache\") pod \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\" (UID: \"61a3a6a7-e308-4fbe-8d2d-13038a63790d\") "
Oct 31 08:01:55.873009 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.872972    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tmp-dir" pod "61a3a6a7-e308-4fbe-8d2d-13038a63790d" (UID: "61a3a6a7-e308-4fbe-8d2d-13038a63790d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:01:55.873201 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.873178    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-model-cache" pod "61a3a6a7-e308-4fbe-8d2d-13038a63790d" (UID: "61a3a6a7-e308-4fbe-8d2d-13038a63790d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:01:55.873257 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.873173    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-home" pod "61a3a6a7-e308-4fbe-8d2d-13038a63790d" (UID: "61a3a6a7-e308-4fbe-8d2d-13038a63790d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:01:55.873346 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.873329    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:01:55.873399 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.873353    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:01:55.873399 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.873366    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:01:55.874949 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.874921    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-dshm" pod "61a3a6a7-e308-4fbe-8d2d-13038a63790d" (UID: "61a3a6a7-e308-4fbe-8d2d-13038a63790d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:01:55.875049 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.874974    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a3a6a7-e308-4fbe-8d2d-13038a63790d-kube-api-access-flx4f" pod "61a3a6a7-e308-4fbe-8d2d-13038a63790d" (UID: "61a3a6a7-e308-4fbe-8d2d-13038a63790d"). InnerVolumeSpecName "kube-api-access-flx4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:01:55.875049 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.874984    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tls-certs" pod "61a3a6a7-e308-4fbe-8d2d-13038a63790d" (UID: "61a3a6a7-e308-4fbe-8d2d-13038a63790d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:01:55.974330 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.974261    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flx4f\" (UniqueName: \"kubernetes.io/projected/61a3a6a7-e308-4fbe-8d2d-13038a63790d-kube-api-access-flx4f\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:01:55.974330 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.974287    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61a3a6a7-e308-4fbe-8d2d-13038a63790d-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:01:55.974330 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:55.974300    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61a3a6a7-e308-4fbe-8d2d-13038a63790d-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:01:56.579185 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.579150    4532 generic.go:356] "Generic (PLEG): container finished" podID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" containerID="c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45" exitCode=0
Oct 31 08:01:56.579621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.579195    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"
Oct 31 08:01:56.579621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.579227    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" event={"ID":"61a3a6a7-e308-4fbe-8d2d-13038a63790d","Type":"ContainerDied","Data":"c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45"}
Oct 31 08:01:56.579621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.579277    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh" event={"ID":"61a3a6a7-e308-4fbe-8d2d-13038a63790d","Type":"ContainerDied","Data":"0a0ba6758fa902d9648bd46c9cc32b43dbf29bcce46537202278688ee176f21c"}
Oct 31 08:01:56.579621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.579299    4532 scope.go:122] "RemoveContainer" containerID="c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45"
Oct 31 08:01:56.587956 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.587940    4532 scope.go:122] "RemoveContainer" containerID="c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45"
Oct 31 08:01:56.588209 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:01:56.588189    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45\": container with ID starting with c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45 not found: ID does not exist" containerID="c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45"
Oct 31 08:01:56.588275 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.588217    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45"} err="failed to get container status \"c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45\": rpc error: code = NotFound desc = could not find container \"c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45\": container with ID starting with c8d9a9e73e08b2dc64683af221d88720ef61f3e7b3e8c270edc51f3b2f8d9a45 not found: ID does not exist"
Oct 31 08:01:56.740032 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.740007    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"]
Oct 31 08:01:56.789299 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:56.789275    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-66d58bcdf9-frzfh"]
Oct 31 08:01:57.624324 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:01:57.624293    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" path="/var/lib/kubelet/pods/61a3a6a7-e308-4fbe-8d2d-13038a63790d/volumes"
Oct 31 08:02:09.310904 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:02:09.310869    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:02:39.309957 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:02:39.309921    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:03:09.310346 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:03:09.310311    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:03:39.310072 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:03:39.310031    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:04:09.310550 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:04:09.310509    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:04:28.620545 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.608862    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"]
Oct 31 08:04:28.643689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.621632    4532 cpu_manager.go:414] "containerMap: removing container" podUID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" containerName="main"
Oct 31 08:04:28.643689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.621658    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" containerName="main"
Oct 31 08:04:28.644227 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.644206    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="61a3a6a7-e308-4fbe-8d2d-13038a63790d" containerName="main"
Oct 31 08:04:28.648301 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.648279    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.780530 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.753213    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"]
Oct 31 08:04:28.808046 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.781840    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:04:28.808299 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.781906    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"seaweedfs-s3-creds\""
Oct 31 08:04:28.808403 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.781988    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-6t9jp\""
Oct 31 08:04:28.833803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.782081    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Oct 31 08:04:28.834051 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.782137    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Oct 31 08:04:28.834160 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.782206    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Oct 31 08:04:28.834261 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.807894    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs7x9\" (UniqueName: \"kubernetes.io/projected/28a60fa9-e874-4249-9102-a0a58743a87f-kube-api-access-rs7x9\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.834452 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.834433    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-home\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.834643 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.834628    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/28a60fa9-e874-4249-9102-a0a58743a87f-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.834795 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.834780    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a60fa9-e874-4249-9102-a0a58743a87f-tls-certs\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.834990 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.834973    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.838485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.835142    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-model-cache\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.838485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.835207    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.838485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.835249    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-dshm\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.935972 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.935877    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-home\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.936211 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.936189    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/28a60fa9-e874-4249-9102-a0a58743a87f-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.936330 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.936316    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a60fa9-e874-4249-9102-a0a58743a87f-tls-certs\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.936504 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.936486    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.936643 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.936625    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-model-cache\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.936794 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.936777    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.936914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.936900    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-dshm\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.937046 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.937032    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs7x9\" (UniqueName: \"kubernetes.io/projected/28a60fa9-e874-4249-9102-a0a58743a87f-kube-api-access-rs7x9\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.937910 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.937880    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-home\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.938698 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.938676    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/28a60fa9-e874-4249-9102-a0a58743a87f-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.944116 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.939061    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-model-cache\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.944246 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.939752    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.944334 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.943838    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.972216 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.972142    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-dshm\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:28.978166 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:28.978101    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a60fa9-e874-4249-9102-a0a58743a87f-tls-certs\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:29.012486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:29.009818    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs7x9\" (UniqueName: \"kubernetes.io/projected/28a60fa9-e874-4249-9102-a0a58743a87f-kube-api-access-rs7x9\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:29.018935 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:29.018647    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:29.971440 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:29.971399    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"]
Oct 31 08:04:30.031383 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:30.028877    4532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 08:04:30.264120 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:30.264005    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" event={"ID":"28a60fa9-e874-4249-9102-a0a58743a87f","Type":"ContainerStarted","Data":"c2e74ad1ae65adee3ff66834dd94b91e0146d06e7f8c0896e5de06cbd08f4d43"}
Oct 31 08:04:38.355701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:38.355664    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" event={"ID":"28a60fa9-e874-4249-9102-a0a58743a87f","Type":"ContainerStarted","Data":"bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5"}
Oct 31 08:04:39.320210 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:04:39.320148    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:04:40.364428 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:40.364393    4532 generic.go:356] "Generic (PLEG): container finished" podID="28a60fa9-e874-4249-9102-a0a58743a87f" containerID="bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5" exitCode=0
Oct 31 08:04:40.365000 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:40.364489    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" event={"ID":"28a60fa9-e874-4249-9102-a0a58743a87f","Type":"ContainerDied","Data":"bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5"}
Oct 31 08:04:41.370396 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:41.370363    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" event={"ID":"28a60fa9-e874-4249-9102-a0a58743a87f","Type":"ContainerStarted","Data":"abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10"}
Oct 31 08:04:41.536952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:41.536905    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" podStartSLOduration=4.036444213 podStartE2EDuration="13.536891116s" totalImagesPullingTime="7.347220472s" totalInitContainerRuntime="2.153226431s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:04:41.535304677 +0000 UTC m=+2132.037458468" watchObservedRunningTime="2026-10-31 08:04:41.536891116 +0000 UTC m=+2132.039044899"
Oct 31 08:04:42.020062 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:42.020034    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:49.019625 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:49.019586    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:49.031740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:49.031717    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:04:49.414481 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:04:49.414441    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:05:09.310399 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:05:09.310363    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:05:25.717405 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:25.717372    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"]
Oct 31 08:05:25.727127 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:05:25.727071    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10" cmd=["/bin/sleep","15"]
Oct 31 08:05:25.727294 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:25.727152    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:05:25.727294 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:05:25.727166    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" containerName="main" containerID="cri-o://abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10"
Oct 31 08:05:25.727294 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:25.727218    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" containerName="main" containerID="cri-o://abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10" gracePeriod=60
Oct 31 08:05:26.070366 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.070335    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:05:26.228889 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.228862    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28a60fa9-e874-4249-9102-a0a58743a87f-kube-api-access-rs7x9\" (UniqueName: \"kubernetes.io/projected/28a60fa9-e874-4249-9102-a0a58743a87f-kube-api-access-rs7x9\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229041 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.228895    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28a60fa9-e874-4249-9102-a0a58743a87f-tls-certs\" (UniqueName: \"kubernetes.io/secret/28a60fa9-e874-4249-9102-a0a58743a87f-tls-certs\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229041 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.228944    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-dshm\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229041 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229011    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-model-cache\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229219 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229128    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28a60fa9-e874-4249-9102-a0a58743a87f-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/28a60fa9-e874-4249-9102-a0a58743a87f-cabundle-cert\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229219 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229175    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-kserve-provision-location\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229327 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229223    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-home\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-home\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229327 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229241    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-model-cache" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:05:26.229327 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229257    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-tmp-dir\") pod \"28a60fa9-e874-4249-9102-a0a58743a87f\" (UID: \"28a60fa9-e874-4249-9102-a0a58743a87f\") "
Oct 31 08:05:26.229547 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229510    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a60fa9-e874-4249-9102-a0a58743a87f-cabundle-cert" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:05:26.229634 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229616    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.229698 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229610    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-tmp-dir" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:05:26.229804 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.229784    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-home" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:05:26.231414 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.231383    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-dshm" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:05:26.231687 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.231668    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a60fa9-e874-4249-9102-a0a58743a87f-kube-api-access-rs7x9" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "kube-api-access-rs7x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:05:26.232100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.232078    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a60fa9-e874-4249-9102-a0a58743a87f-tls-certs" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:05:26.247942 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.247889    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-kserve-provision-location" pod "28a60fa9-e874-4249-9102-a0a58743a87f" (UID: "28a60fa9-e874-4249-9102-a0a58743a87f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:05:26.330853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.330830    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/28a60fa9-e874-4249-9102-a0a58743a87f-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.330853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.330851    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.330983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.330862    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.330983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.330890    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.330983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.330898    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs7x9\" (UniqueName: \"kubernetes.io/projected/28a60fa9-e874-4249-9102-a0a58743a87f-kube-api-access-rs7x9\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.330983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.330907    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a60fa9-e874-4249-9102-a0a58743a87f-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.330983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.330916    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28a60fa9-e874-4249-9102-a0a58743a87f-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:05:26.554711 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.554683    4532 generic.go:356] "Generic (PLEG): container finished" podID="28a60fa9-e874-4249-9102-a0a58743a87f" containerID="abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10" exitCode=0
Oct 31 08:05:26.554830 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.554723    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" event={"ID":"28a60fa9-e874-4249-9102-a0a58743a87f","Type":"ContainerDied","Data":"abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10"}
Oct 31 08:05:26.554830 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.554737    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"
Oct 31 08:05:26.554830 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.554762    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s" event={"ID":"28a60fa9-e874-4249-9102-a0a58743a87f","Type":"ContainerDied","Data":"c2e74ad1ae65adee3ff66834dd94b91e0146d06e7f8c0896e5de06cbd08f4d43"}
Oct 31 08:05:26.554830 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.554783    4532 scope.go:122] "RemoveContainer" containerID="abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10"
Oct 31 08:05:26.563409 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.563392    4532 scope.go:122] "RemoveContainer" containerID="bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5"
Oct 31 08:05:26.570430 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.570412    4532 scope.go:122] "RemoveContainer" containerID="abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10"
Oct 31 08:05:26.570706 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:05:26.570686    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10\": container with ID starting with abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10 not found: ID does not exist" containerID="abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10"
Oct 31 08:05:26.570766 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.570714    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10"} err="failed to get container status \"abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10\": rpc error: code = NotFound desc = could not find container \"abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10\": container with ID starting with abf0a1b050e4e1a1cc8598b4aee254053d3a642c38616d28a705c773b7eecb10 not found: ID does not exist"
Oct 31 08:05:26.570766 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.570733    4532 scope.go:122] "RemoveContainer" containerID="bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5"
Oct 31 08:05:26.570992 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:05:26.570973    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5\": container with ID starting with bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5 not found: ID does not exist" containerID="bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5"
Oct 31 08:05:26.571033 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.570999    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5"} err="failed to get container status \"bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5\": rpc error: code = NotFound desc = could not find container \"bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5\": container with ID starting with bf582fa4de5810007cda2143357e8cac88f98b743da849ba914bcd3409045da5 not found: ID does not exist"
Oct 31 08:05:26.716127 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.716103    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"]
Oct 31 08:05:26.719266 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:26.719240    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-gkr6s"]
Oct 31 08:05:27.624632 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:05:27.624601    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" path="/var/lib/kubelet/pods/28a60fa9-e874-4249-9102-a0a58743a87f/volumes"
Oct 31 08:05:39.309749 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:05:39.309714    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:06:09.310481 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:06:09.310430    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:06:20.536100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.536065    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"]
Oct 31 08:06:20.536703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.536555    4532 cpu_manager.go:414] "containerMap: removing container" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" containerName="main"
Oct 31 08:06:20.536703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.536577    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" containerName="main"
Oct 31 08:06:20.536703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.536604    4532 cpu_manager.go:414] "containerMap: removing container" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" containerName="storage-initializer"
Oct 31 08:06:20.536703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.536612    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" containerName="storage-initializer"
Oct 31 08:06:20.536936 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.536713    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="28a60fa9-e874-4249-9102-a0a58743a87f" containerName="main"
Oct 31 08:06:20.540776 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.540755    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.592696 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.592671    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"seaweedfs-s3-creds\""
Oct 31 08:06:20.641855 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.641824    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Oct 31 08:06:20.641855 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.641839    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-s4gtz\""
Oct 31 08:06:20.641855 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.641833    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Oct 31 08:06:20.642043 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.641835    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Oct 31 08:06:20.642043 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.641881    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:06:20.645139 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.645117    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"]
Oct 31 08:06:20.652428 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652407    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-home\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.652660 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652444    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.652660 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652573    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-model-cache\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.652660 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652598    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/386516e0-acfc-4e19-a17e-d42723726a30-tls-certs\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.652892 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652756    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/386516e0-acfc-4e19-a17e-d42723726a30-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.652892 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652793    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgrtn\" (UniqueName: \"kubernetes.io/projected/386516e0-acfc-4e19-a17e-d42723726a30-kube-api-access-qgrtn\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.652892 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652851    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.653003 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.652900    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-dshm\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.753824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.753793    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-dshm\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.753844    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-home\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.753860    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.753886    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-model-cache\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.753905    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/386516e0-acfc-4e19-a17e-d42723726a30-tls-certs\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754248 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754038    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/386516e0-acfc-4e19-a17e-d42723726a30-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754248 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754078    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgrtn\" (UniqueName: \"kubernetes.io/projected/386516e0-acfc-4e19-a17e-d42723726a30-kube-api-access-qgrtn\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754248 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754108    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754402 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754284    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-home\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754402 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754371    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-model-cache\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754524 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754402    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754524 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754491    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.754880 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.754863    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/386516e0-acfc-4e19-a17e-d42723726a30-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.756163 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.756144    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-dshm\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.756510 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.756494    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/386516e0-acfc-4e19-a17e-d42723726a30-tls-certs\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.810001 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.809931    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgrtn\" (UniqueName: \"kubernetes.io/projected/386516e0-acfc-4e19-a17e-d42723726a30-kube-api-access-qgrtn\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-28d66\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:20.855056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:20.855030    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:21.281929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:21.280059    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"]
Oct 31 08:06:21.285035 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:06:21.284993    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod386516e0_acfc_4e19_a17e_d42723726a30.slice/crio-00e966e75bc49646a351f4c0b49814b0e6b79c4171f0ce729a90274d66893784 WatchSource:0}: Error finding container 00e966e75bc49646a351f4c0b49814b0e6b79c4171f0ce729a90274d66893784: Status 404 returned error can't find the container with id 00e966e75bc49646a351f4c0b49814b0e6b79c4171f0ce729a90274d66893784
Oct 31 08:06:21.773942 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:21.773904    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" event={"ID":"386516e0-acfc-4e19-a17e-d42723726a30","Type":"ContainerStarted","Data":"b9ef8c243067b48c2093732339282198c366c92577b5213e770afbfac2b71da6"}
Oct 31 08:06:21.775334 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:21.775306    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" event={"ID":"386516e0-acfc-4e19-a17e-d42723726a30","Type":"ContainerStarted","Data":"00e966e75bc49646a351f4c0b49814b0e6b79c4171f0ce729a90274d66893784"}
Oct 31 08:06:23.785718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:23.785684    4532 generic.go:356] "Generic (PLEG): container finished" podID="386516e0-acfc-4e19-a17e-d42723726a30" containerID="b9ef8c243067b48c2093732339282198c366c92577b5213e770afbfac2b71da6" exitCode=0
Oct 31 08:06:23.786077 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:23.785720    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" event={"ID":"386516e0-acfc-4e19-a17e-d42723726a30","Type":"ContainerDied","Data":"b9ef8c243067b48c2093732339282198c366c92577b5213e770afbfac2b71da6"}
Oct 31 08:06:24.792727 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:24.792687    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" event={"ID":"386516e0-acfc-4e19-a17e-d42723726a30","Type":"ContainerStarted","Data":"16fd1b701b4d54f7074aae7b534a78553a6ab4fb192725c3ff8beed48eeaba72"}
Oct 31 08:06:24.860592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:24.860541    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:24.972487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:24.971892    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" podStartSLOduration=2.686113398 podStartE2EDuration="4.971869372s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.285755974s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:06:24.915987706 +0000 UTC m=+2235.418141490" watchObservedRunningTime="2026-10-31 08:06:24.971869372 +0000 UTC m=+2235.474023163"
Oct 31 08:06:30.855876 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:30.855839    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:30.868006 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:30.867985    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:31.831506 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:06:31.831476    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:06:39.310332 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:06:39.310289    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:07:09.311026 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:07:09.310990    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:07:13.684546 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:13.684512    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"]
Oct 31 08:07:13.695301 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:07:13.695266    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="16fd1b701b4d54f7074aae7b534a78553a6ab4fb192725c3ff8beed48eeaba72" cmd=["/bin/sleep","15"]
Oct 31 08:07:13.695457 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:13.695316    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:07:13.695457 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:07:13.695329    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="main" containerID="cri-o://16fd1b701b4d54f7074aae7b534a78553a6ab4fb192725c3ff8beed48eeaba72"
Oct 31 08:07:13.695457 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:13.695362    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="main" containerID="cri-o://16fd1b701b4d54f7074aae7b534a78553a6ab4fb192725c3ff8beed48eeaba72" gracePeriod=60
Oct 31 08:07:13.821196 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:13.821165    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/health\": dial tcp 10.133.0.35:8000: connect: connection refused"
Oct 31 08:07:13.987315 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:13.987285    4532 generic.go:356] "Generic (PLEG): container finished" podID="386516e0-acfc-4e19-a17e-d42723726a30" containerID="16fd1b701b4d54f7074aae7b534a78553a6ab4fb192725c3ff8beed48eeaba72" exitCode=0
Oct 31 08:07:13.987452 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:13.987354    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" event={"ID":"386516e0-acfc-4e19-a17e-d42723726a30","Type":"ContainerDied","Data":"16fd1b701b4d54f7074aae7b534a78553a6ab4fb192725c3ff8beed48eeaba72"}
Oct 31 08:07:14.047934 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.047906    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:07:14.080527 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080498    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-home\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-home\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.080701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080542    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/386516e0-acfc-4e19-a17e-d42723726a30-tls-certs\" (UniqueName: \"kubernetes.io/secret/386516e0-acfc-4e19-a17e-d42723726a30-tls-certs\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.080701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080586    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-tmp-dir\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.080701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080629    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-dshm\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-dshm\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.080701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080677    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/386516e0-acfc-4e19-a17e-d42723726a30-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/386516e0-acfc-4e19-a17e-d42723726a30-cabundle-cert\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.080909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080718    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/386516e0-acfc-4e19-a17e-d42723726a30-kube-api-access-qgrtn\" (UniqueName: \"kubernetes.io/projected/386516e0-acfc-4e19-a17e-d42723726a30-kube-api-access-qgrtn\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.080909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080765    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-home" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:07:14.080909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080814    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-model-cache\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.080909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080837    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-tmp-dir" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:07:14.080909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080861    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-kserve-provision-location\") pod \"386516e0-acfc-4e19-a17e-d42723726a30\" (UID: \"386516e0-acfc-4e19-a17e-d42723726a30\") "
Oct 31 08:07:14.081162 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.080994    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-model-cache" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:07:14.081162 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.081022    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/386516e0-acfc-4e19-a17e-d42723726a30-cabundle-cert" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:07:14.081261 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.081165    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.081261 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.081191    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.081261 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.081209    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.081261 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.081222    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/386516e0-acfc-4e19-a17e-d42723726a30-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.082904 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.082879    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-dshm" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:07:14.083015 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.082921    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/386516e0-acfc-4e19-a17e-d42723726a30-kube-api-access-qgrtn" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "kube-api-access-qgrtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:07:14.083015 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.082968    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/386516e0-acfc-4e19-a17e-d42723726a30-tls-certs" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:07:14.099523 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.099495    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-kserve-provision-location" pod "386516e0-acfc-4e19-a17e-d42723726a30" (UID: "386516e0-acfc-4e19-a17e-d42723726a30"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:07:14.182550 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.182529    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/386516e0-acfc-4e19-a17e-d42723726a30-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.182550 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.182550    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.182691 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.182560    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgrtn\" (UniqueName: \"kubernetes.io/projected/386516e0-acfc-4e19-a17e-d42723726a30-kube-api-access-qgrtn\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.182691 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.182570    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/386516e0-acfc-4e19-a17e-d42723726a30-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:07:14.993219 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.993190    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"
Oct 31 08:07:14.993674 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.993190    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66" event={"ID":"386516e0-acfc-4e19-a17e-d42723726a30","Type":"ContainerDied","Data":"00e966e75bc49646a351f4c0b49814b0e6b79c4171f0ce729a90274d66893784"}
Oct 31 08:07:14.993674 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:14.993348    4532 scope.go:122] "RemoveContainer" containerID="16fd1b701b4d54f7074aae7b534a78553a6ab4fb192725c3ff8beed48eeaba72"
Oct 31 08:07:15.002375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:15.002360    4532 scope.go:122] "RemoveContainer" containerID="b9ef8c243067b48c2093732339282198c366c92577b5213e770afbfac2b71da6"
Oct 31 08:07:15.204998 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:15.204967    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"]
Oct 31 08:07:15.210072 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:15.210049    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-28d66"]
Oct 31 08:07:15.624363 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:07:15.624335    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="386516e0-acfc-4e19-a17e-d42723726a30" path="/var/lib/kubelet/pods/386516e0-acfc-4e19-a17e-d42723726a30/volumes"
Oct 31 08:07:39.311040 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:07:39.311003    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:08:09.310088 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:08:09.310050    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:08:39.310640 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:08:39.310594    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:09:09.310854 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:09:09.310820    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:09:39.310682 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:09:39.310646    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:09:57.154500 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.154454    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"]
Oct 31 08:09:57.155063 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.154679    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" containerID="cri-o://6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805" gracePeriod=30
Oct 31 08:09:57.203782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.203751    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"]
Oct 31 08:09:57.203968 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.203934    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy" containerID="cri-o://9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710" gracePeriod=30
Oct 31 08:09:57.204734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.204706    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"]
Oct 31 08:09:57.204880 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.204860    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy" containerID="cri-o://e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f" gracePeriod=30
Oct 31 08:09:57.253538 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.253511    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"]
Oct 31 08:09:57.253764 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:57.253742    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy" containerID="cri-o://32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869" gracePeriod=30
Oct 31 08:09:58.069282 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:58.069245    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:09:58.069450 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:58.069316    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:09:59.003685 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:59.003642    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:09:59.004052 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:59.003723    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:09:59.047205 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:59.047177    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:09:59.047327 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:09:59.047218    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:10:00.949434 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:00.949402    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:10:00.949853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:00.949448    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:10:02.442678 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.442645    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 08:10:02.490275 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.490251    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 08:10:02.535826 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.535800    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 08:10:02.542356 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.542336    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 08:10:02.566686 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566659    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-credential-socket\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.566779 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566691    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-istio-token\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-istio-token\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.566779 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566715    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-socket\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.566779 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566734    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-data\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.566779 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566767    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09802de6-13c5-40c7-a219-42515a5195fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09802de6-13c5-40c7-a219-42515a5195fb-istiod-ca-cert\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.566960 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566788    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-envoy\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.566960 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566839    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/09802de6-13c5-40c7-a219-42515a5195fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09802de6-13c5-40c7-a219-42515a5195fb-istio-podinfo\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.567056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566925    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-credential-socket" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.567056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.566979    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-socket" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.567056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567016    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-kube-api-access-wfk26\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-kube-api-access-wfk26\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.567221 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567069    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-certs\") pod \"09802de6-13c5-40c7-a219-42515a5195fb\" (UID: \"09802de6-13c5-40c7-a219-42515a5195fb\") "
Oct 31 08:10:02.567221 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567160    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-data" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.567347 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567309    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09802de6-13c5-40c7-a219-42515a5195fb-istiod-ca-cert" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:10:02.567437 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567408    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-certs" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.567545 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567522    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.567584 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567549    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.567584 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567563    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09802de6-13c5-40c7-a219-42515a5195fb-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.567584 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.567578    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.569039 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.569013    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-istio-token" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.569143 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.569045    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09802de6-13c5-40c7-a219-42515a5195fb-istio-podinfo" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:10:02.569143 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.569054    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-envoy" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.569254 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.569194    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-kube-api-access-wfk26" pod "09802de6-13c5-40c7-a219-42515a5195fb" (UID: "09802de6-13c5-40c7-a219-42515a5195fb"). InnerVolumeSpecName "kube-api-access-wfk26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.647260 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.647227    4532 generic.go:356] "Generic (PLEG): container finished" podID="09802de6-13c5-40c7-a219-42515a5195fb" containerID="6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805" exitCode=0
Oct 31 08:10:02.647395 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.647278    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"
Oct 31 08:10:02.647395 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.647307    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" event={"ID":"09802de6-13c5-40c7-a219-42515a5195fb","Type":"ContainerDied","Data":"6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805"}
Oct 31 08:10:02.647395 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.647358    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2" event={"ID":"09802de6-13c5-40c7-a219-42515a5195fb","Type":"ContainerDied","Data":"88bd96164d0b7380ba946dbb21d76392a30e06c3c34804d1ac74ad043936efbb"}
Oct 31 08:10:02.647395 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.647382    4532 scope.go:122] "RemoveContainer" containerID="6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805"
Oct 31 08:10:02.648673 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.648650    4532 generic.go:356] "Generic (PLEG): container finished" podID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerID="32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869" exitCode=0
Oct 31 08:10:02.648796 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.648688    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"
Oct 31 08:10:02.648796 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.648723    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" event={"ID":"ab696e69-e4bd-4141-bc1d-116c1490b720","Type":"ContainerDied","Data":"32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869"}
Oct 31 08:10:02.648796 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.648757    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls" event={"ID":"ab696e69-e4bd-4141-bc1d-116c1490b720","Type":"ContainerDied","Data":"4bc9b08e3321a4980b11955637e069cb219fb9abc6e0d13eaa1cc14cd952325f"}
Oct 31 08:10:02.650197 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.650173    4532 generic.go:356] "Generic (PLEG): container finished" podID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerID="9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710" exitCode=0
Oct 31 08:10:02.650314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.650196    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" event={"ID":"3b0b3541-527a-4f06-ac6b-c34c9fb8981e","Type":"ContainerDied","Data":"9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710"}
Oct 31 08:10:02.650314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.650209    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"
Oct 31 08:10:02.650314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.650240    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm" event={"ID":"3b0b3541-527a-4f06-ac6b-c34c9fb8981e","Type":"ContainerDied","Data":"5211e0bc38ba3c55abd95c35321b99b5f430ec7bedc9b231900a5f3f0f03995a"}
Oct 31 08:10:02.651538 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.651518    4532 generic.go:356] "Generic (PLEG): container finished" podID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerID="e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f" exitCode=0
Oct 31 08:10:02.651637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.651554    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"
Oct 31 08:10:02.651637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.651562    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" event={"ID":"1aca2a6b-5d08-4fc4-be07-24cf10901c4a","Type":"ContainerDied","Data":"e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f"}
Oct 31 08:10:02.651637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.651587    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw" event={"ID":"1aca2a6b-5d08-4fc4-be07-24cf10901c4a","Type":"ContainerDied","Data":"af7c1f3fc04750663a252a018c619b002e22a252bcb4e919fb9f3c303b83124e"}
Oct 31 08:10:02.655839 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.655811    4532 scope.go:122] "RemoveContainer" containerID="6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805"
Oct 31 08:10:02.656405 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:10:02.656196    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805\": container with ID starting with 6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805 not found: ID does not exist" containerID="6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805"
Oct 31 08:10:02.656405 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.656238    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805"} err="failed to get container status \"6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805\": rpc error: code = NotFound desc = could not find container \"6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805\": container with ID starting with 6c99925b4aa28b1ebf63afc8b6046c14dd72e81ae153977f42dce8310f512805 not found: ID does not exist"
Oct 31 08:10:02.656405 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.656260    4532 scope.go:122] "RemoveContainer" containerID="32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869"
Oct 31 08:10:02.663428 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.663414    4532 scope.go:122] "RemoveContainer" containerID="32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869"
Oct 31 08:10:02.663693 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:10:02.663670    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869\": container with ID starting with 32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869 not found: ID does not exist" containerID="32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869"
Oct 31 08:10:02.663745 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.663698    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869"} err="failed to get container status \"32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869\": rpc error: code = NotFound desc = could not find container \"32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869\": container with ID starting with 32edf929661107d1f1b534b08e56f5f6cec9332200557654feb2b2c1f0401869 not found: ID does not exist"
Oct 31 08:10:02.663745 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.663715    4532 scope.go:122] "RemoveContainer" containerID="9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710"
Oct 31 08:10:02.668714 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668692    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab696e69-e4bd-4141-bc1d-116c1490b720-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab696e69-e4bd-4141-bc1d-116c1490b720-istiod-ca-cert\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.668821 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668735    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-podinfo\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.668821 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668765    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-data\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.668821 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668805    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-kube-api-access-2hmjd\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-kube-api-access-2hmjd\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.668977 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668842    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-podinfo\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.668977 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668871    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-certs\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.668977 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668910    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-credential-socket\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.668977 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668943    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-certs\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.669177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.668982    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istiod-ca-cert\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.669177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669011    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-token\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-token\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.669177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669068    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-podinfo\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.669177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669072    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab696e69-e4bd-4141-bc1d-116c1490b720-istiod-ca-cert" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:10:02.669177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669087    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-data" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.669177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669097    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-socket\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.669177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669151    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istiod-ca-cert\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669190    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-token\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-token\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669257    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-envoy\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669286    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-certs\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669313    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-socket\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669342    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-certs" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669350    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-data\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669425    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-envoy\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669475    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-envoy\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669525    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-kube-api-access-hrp87\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-kube-api-access-hrp87\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.669583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669558    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-credential-socket\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669586    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-credential-socket\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669592    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-data" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669624    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-kube-api-access-d6kk9\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-kube-api-access-d6kk9\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669663    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istiod-ca-cert" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669679    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-socket" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669675    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-token\") pod \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\" (UID: \"3b0b3541-527a-4f06-ac6b-c34c9fb8981e\") "
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669736    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-data\") pod \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\" (UID: \"1aca2a6b-5d08-4fc4-be07-24cf10901c4a\") "
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669779    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-socket\") pod \"ab696e69-e4bd-4141-bc1d-116c1490b720\" (UID: \"ab696e69-e4bd-4141-bc1d-116c1490b720\") "
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.669899    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-certs" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.670110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670091    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-socket" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670148    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab696e69-e4bd-4141-bc1d-116c1490b720-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670169    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670186    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09802de6-13c5-40c7-a219-42515a5195fb-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670201    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfk26\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-kube-api-access-wfk26\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670216    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670229    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670244    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670259    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670272    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09802de6-13c5-40c7-a219-42515a5195fb-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670286    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670298    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670311    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09802de6-13c5-40c7-a219-42515a5195fb-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.670653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.670356    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istiod-ca-cert" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:10:02.671383 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.671186    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-podinfo" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:10:02.671537 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.671485    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-kube-api-access-2hmjd" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "kube-api-access-2hmjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.671739 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.671715    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-socket" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.671848 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.671821    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-token" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.671956 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.671937    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-data" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.672701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.672133    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-credential-socket" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.672701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.672296    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-token" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.672701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.672340    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-credential-socket" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.672701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.672532    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-podinfo" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:10:02.672701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.672574    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-credential-socket" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.672701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.672670    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-certs" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.673195    4532 scope.go:122] "RemoveContainer" containerID="9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710"
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.673192    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-token" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.673534    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-envoy" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:10:02.673575    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710\": container with ID starting with 9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710 not found: ID does not exist" containerID="9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710"
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.673608    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710"} err="failed to get container status \"9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710\": rpc error: code = NotFound desc = could not find container \"9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710\": container with ID starting with 9825a827104becfb6db0d083bb5b9dd8e89487fc2ccd42965dc0ba791d82b710 not found: ID does not exist"
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.673632    4532 scope.go:122] "RemoveContainer" containerID="e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f"
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.673766    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-podinfo" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:10:02.673993 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.673922    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-envoy" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.674626 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.674600    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-kube-api-access-d6kk9" pod "1aca2a6b-5d08-4fc4-be07-24cf10901c4a" (UID: "1aca2a6b-5d08-4fc4-be07-24cf10901c4a"). InnerVolumeSpecName "kube-api-access-d6kk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.674726 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.674676    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-envoy" pod "3b0b3541-527a-4f06-ac6b-c34c9fb8981e" (UID: "3b0b3541-527a-4f06-ac6b-c34c9fb8981e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:10:02.674963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.674944    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-kube-api-access-hrp87" pod "ab696e69-e4bd-4141-bc1d-116c1490b720" (UID: "ab696e69-e4bd-4141-bc1d-116c1490b720"). InnerVolumeSpecName "kube-api-access-hrp87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:10:02.680762 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.680742    4532 scope.go:122] "RemoveContainer" containerID="e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f"
Oct 31 08:10:02.681012 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:10:02.680993    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f\": container with ID starting with e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f not found: ID does not exist" containerID="e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f"
Oct 31 08:10:02.681078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.681016    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f"} err="failed to get container status \"e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f\": rpc error: code = NotFound desc = could not find container \"e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f\": container with ID starting with e855e40b92f0128757374c2ba63feec3dc11ae54992e72d91370c966c8401f7f not found: ID does not exist"
Oct 31 08:10:02.770790 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770761    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hmjd\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-kube-api-access-2hmjd\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770790 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770783    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770790 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770793    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770802    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770811    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770820    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770828    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770837    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770845    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770853    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770862    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770871    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770880    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrp87\" (UniqueName: \"kubernetes.io/projected/ab696e69-e4bd-4141-bc1d-116c1490b720-kube-api-access-hrp87\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770888    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770896    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770905    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6kk9\" (UniqueName: \"kubernetes.io/projected/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-kube-api-access-d6kk9\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770913    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770921    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aca2a6b-5d08-4fc4-be07-24cf10901c4a-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770930    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab696e69-e4bd-4141-bc1d-116c1490b720-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:02.770978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:02.770939    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b3541-527a-4f06-ac6b-c34c9fb8981e-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:10:03.112620 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.112580    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"]
Oct 31 08:10:03.117255 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.117215    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-cs2m2"]
Oct 31 08:10:03.329680 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.329655    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"]
Oct 31 08:10:03.333832 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.333809    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-tgnls"]
Oct 31 08:10:03.440939 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.440914    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"]
Oct 31 08:10:03.446668 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.446647    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-wjnpw"]
Oct 31 08:10:03.553907 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.553879    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"]
Oct 31 08:10:03.557385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.557358    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-2wsfm"]
Oct 31 08:10:03.624244 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.624188    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09802de6-13c5-40c7-a219-42515a5195fb" path="/var/lib/kubelet/pods/09802de6-13c5-40c7-a219-42515a5195fb/volumes"
Oct 31 08:10:03.624611 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.624597    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" path="/var/lib/kubelet/pods/1aca2a6b-5d08-4fc4-be07-24cf10901c4a/volumes"
Oct 31 08:10:03.624980 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.624968    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" path="/var/lib/kubelet/pods/3b0b3541-527a-4f06-ac6b-c34c9fb8981e/volumes"
Oct 31 08:10:03.625666 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:03.625652    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" path="/var/lib/kubelet/pods/ab696e69-e4bd-4141-bc1d-116c1490b720/volumes"
Oct 31 08:10:09.310090 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:10:09.310057    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:10:21.904515 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904431    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"]
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904760    4532 cpu_manager.go:414] "containerMap: removing container" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="storage-initializer"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904773    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="storage-initializer"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904790    4532 cpu_manager.go:414] "containerMap: removing container" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904795    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904815    4532 cpu_manager.go:414] "containerMap: removing container" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904819    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904827    4532 cpu_manager.go:414] "containerMap: removing container" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904832    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904840    4532 cpu_manager.go:414] "containerMap: removing container" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904844    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904852    4532 cpu_manager.go:414] "containerMap: removing container" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="main"
Oct 31 08:10:21.904866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904856    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="main"
Oct 31 08:10:21.905205 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904917    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="1aca2a6b-5d08-4fc4-be07-24cf10901c4a" containerName="istio-proxy"
Oct 31 08:10:21.905205 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904924    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab696e69-e4bd-4141-bc1d-116c1490b720" containerName="istio-proxy"
Oct 31 08:10:21.905205 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904930    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b0b3541-527a-4f06-ac6b-c34c9fb8981e" containerName="istio-proxy"
Oct 31 08:10:21.905205 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904936    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="386516e0-acfc-4e19-a17e-d42723726a30" containerName="main"
Oct 31 08:10:21.905205 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.904942    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="09802de6-13c5-40c7-a219-42515a5195fb" containerName="istio-proxy"
Oct 31 08:10:21.908916 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.908899    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:21.961059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.961023    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:10:21.989388 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.989358    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Oct 31 08:10:21.989388 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:21.989358    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Oct 31 08:10:22.010016 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.009994    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"seaweedfs-s3-creds\""
Oct 31 08:10:22.010129 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.010107    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Oct 31 08:10:22.010187 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.010131    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-dpjc7\""
Oct 31 08:10:22.011407 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.011383    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"]
Oct 31 08:10:22.102746 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102722    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47f8c\" (UniqueName: \"kubernetes.io/projected/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kube-api-access-47f8c\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.102882 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102755    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.102882 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102775    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-dshm\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.102882 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102791    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-model-cache\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.102882 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102851    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.103032 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102916    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fa17e96d-7f23-4556-96f6-79ee2db47fa7-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.103032 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102949    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-home\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.103032 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.102965    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tls-certs\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204081 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204018    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-home\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204081 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204049    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tls-certs\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204104    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47f8c\" (UniqueName: \"kubernetes.io/projected/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kube-api-access-47f8c\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204135    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204339 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204274    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-dshm\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204339 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204314    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-model-cache\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204434 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204409    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204517 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204449    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fa17e96d-7f23-4556-96f6-79ee2db47fa7-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204517 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204499    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-home\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204517 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204506    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204729 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204701    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-model-cache\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.204839 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.204780    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.205097 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.205077    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fa17e96d-7f23-4556-96f6-79ee2db47fa7-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.206488 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.206451    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-dshm\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.206588 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.206570    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tls-certs\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.261004 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.260977    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47f8c\" (UniqueName: \"kubernetes.io/projected/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kube-api-access-47f8c\") pod \"tokenizer-template-args-test-kserve-6784ddf685-9dpbz\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.523519 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.523431    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:22.944709 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:22.944675    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"]
Oct 31 08:10:22.947447 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:10:22.947418    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa17e96d_7f23_4556_96f6_79ee2db47fa7.slice/crio-62ce7e691ecb79d4f433443d06cc4ca81dbd2bd000761e27d0cf8bae400d0a6a WatchSource:0}: Error finding container 62ce7e691ecb79d4f433443d06cc4ca81dbd2bd000761e27d0cf8bae400d0a6a: Status 404 returned error can't find the container with id 62ce7e691ecb79d4f433443d06cc4ca81dbd2bd000761e27d0cf8bae400d0a6a
Oct 31 08:10:23.730872 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:23.730832    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" event={"ID":"fa17e96d-7f23-4556-96f6-79ee2db47fa7","Type":"ContainerStarted","Data":"081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26"}
Oct 31 08:10:23.731022 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:23.730900    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" event={"ID":"fa17e96d-7f23-4556-96f6-79ee2db47fa7","Type":"ContainerStarted","Data":"62ce7e691ecb79d4f433443d06cc4ca81dbd2bd000761e27d0cf8bae400d0a6a"}
Oct 31 08:10:25.305221 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.305185    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"]
Oct 31 08:10:25.309135 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.309115    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328225 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328199    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-data\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328346 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328240    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-credential-socket\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328387 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328306    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-envoy\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328443 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328429    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-socket\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328505 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328490    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-certs\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328555 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328540    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbvwf\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-kube-api-access-mbvwf\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328660 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328635    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-podinfo\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328698 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328671    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-token\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.328731 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.328721    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.361755 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.361725    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-service-ca.crt\""
Oct 31 08:10:25.395404 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.395375    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-gw-ca-root-cert\""
Oct 31 08:10:25.404388 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.404359    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"kube-root-ca.crt\""
Oct 31 08:10:25.404487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.404452    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"router-gateway-1-openshift-default-dockercfg-8mcqm\""
Oct 31 08:10:25.411255 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.411230    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"]
Oct 31 08:10:25.429206 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429176    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-podinfo\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429206 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429208    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-token\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429225    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429249    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-data\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429299    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-credential-socket\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429316    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-envoy\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429352    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-socket\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429638 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429391    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-certs\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429638 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429420    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbvwf\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-kube-api-access-mbvwf\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429765 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429739    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-data\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429827 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429789    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-credential-socket\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.429967 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429947    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.430040 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.429981    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-socket\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.430085 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.430042    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-certs\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.431649 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.431628    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-envoy\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.431774 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.431759    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-podinfo\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.486260 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.486233    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-token\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.486364 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.486348    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbvwf\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-kube-api-access-mbvwf\") pod \"router-gateway-1-openshift-default-764c68896-g7tm2\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") " pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.621214 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.621122    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:25.743017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.742988    4532 generic.go:356] "Generic (PLEG): container finished" podID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerID="081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26" exitCode=0
Oct 31 08:10:25.743138 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:25.743046    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" event={"ID":"fa17e96d-7f23-4556-96f6-79ee2db47fa7","Type":"ContainerDied","Data":"081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26"}
Oct 31 08:10:26.043768 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.043615    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"]
Oct 31 08:10:26.046401 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:10:26.046369    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08975cf2_7d59_4040_ac89_9fc6e44b54c3.slice/crio-83eca5447641e2ee0756db02cea68725c15f115dad55d4bb7d815e110523a8cd WatchSource:0}: Error finding container 83eca5447641e2ee0756db02cea68725c15f115dad55d4bb7d815e110523a8cd: Status 404 returned error can't find the container with id 83eca5447641e2ee0756db02cea68725c15f115dad55d4bb7d815e110523a8cd
Oct 31 08:10:26.048430 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.048402    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:26.048523 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.048486    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:26.048523 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.048515    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:26.749293 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.749204    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" event={"ID":"08975cf2-7d59-4040-ac89-9fc6e44b54c3","Type":"ContainerStarted","Data":"886ae0e6c71be9b1ca76b3494093fa25a2ea985dc71b13f0737c3f2ae9409b65"}
Oct 31 08:10:26.749293 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.749267    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" event={"ID":"08975cf2-7d59-4040-ac89-9fc6e44b54c3","Type":"ContainerStarted","Data":"83eca5447641e2ee0756db02cea68725c15f115dad55d4bb7d815e110523a8cd"}
Oct 31 08:10:26.751143 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.751116    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" event={"ID":"fa17e96d-7f23-4556-96f6-79ee2db47fa7","Type":"ContainerStarted","Data":"4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0"}
Oct 31 08:10:26.864353 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.864307    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" podStartSLOduration=1.864294882 podStartE2EDuration="1.864294882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:10:26.862700554 +0000 UTC m=+2477.364854345" watchObservedRunningTime="2026-10-31 08:10:26.864294882 +0000 UTC m=+2477.366448672"
Oct 31 08:10:26.979141 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:26.979088    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" podStartSLOduration=3.27166945 podStartE2EDuration="5.97907478s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.70740533s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:10:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:10:26.978118931 +0000 UTC m=+2477.480272725" watchObservedRunningTime="2026-10-31 08:10:26.97907478 +0000 UTC m=+2477.481228564"
Oct 31 08:10:27.523919 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:27.523874    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:27.624579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:27.624553    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:27.626407 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:27.626385    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:27.755592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:27.755565    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:27.756226 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:27.756208    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:32.524554 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:32.524519    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:32.536691 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:32.536668    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:32.784976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:32.784910    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:10:38.453536 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.453507    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"]
Oct 31 08:10:38.456194 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.456178    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.526384 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.526354    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.526520 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.526394    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.526520 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.526412    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.526520 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.526484    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.526697 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.526532    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncr8v\" (UniqueName: \"kubernetes.io/projected/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-kube-api-access-ncr8v\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.526697 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.526655    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.557478 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.557437    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"default-dockercfg-rtgsb\""
Oct 31 08:10:38.557478 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.557444    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Oct 31 08:10:38.558746 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.558721    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"]
Oct 31 08:10:38.627182 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627150    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627332 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627199    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627379 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627327    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627379 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627364    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627498 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627388    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncr8v\" (UniqueName: \"kubernetes.io/projected/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-kube-api-access-ncr8v\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627498 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627444    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627697 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627667    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627812 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627709    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.627812 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.627764    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.629383 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.629354    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.629879 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.629856    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.683626 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.683601    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncr8v\" (UniqueName: \"kubernetes.io/projected/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-kube-api-access-ncr8v\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:38.767791 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:38.767731    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:39.136675 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:39.136620    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"]
Oct 31 08:10:39.138816 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:10:39.138789    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda897a063_3ddc_4771_9399_fb7bf7ad8a3d.slice/crio-91290d6def1edaadf52a92265b67171c3b26012ee71316397278984a29c1973e WatchSource:0}: Error finding container 91290d6def1edaadf52a92265b67171c3b26012ee71316397278984a29c1973e: Status 404 returned error can't find the container with id 91290d6def1edaadf52a92265b67171c3b26012ee71316397278984a29c1973e
Oct 31 08:10:39.310653 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:10:39.310619    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:10:39.804287 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:39.804250    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" event={"ID":"a897a063-3ddc-4771-9399-fb7bf7ad8a3d","Type":"ContainerStarted","Data":"eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1"}
Oct 31 08:10:39.804722 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:39.804306    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" event={"ID":"a897a063-3ddc-4771-9399-fb7bf7ad8a3d","Type":"ContainerStarted","Data":"91290d6def1edaadf52a92265b67171c3b26012ee71316397278984a29c1973e"}
Oct 31 08:10:39.917777 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:39.917732    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" podStartSLOduration=1.917718626 podStartE2EDuration="1.917718626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:10:39.916723543 +0000 UTC m=+2490.418877332" watchObservedRunningTime="2026-10-31 08:10:39.917718626 +0000 UTC m=+2490.419872415"
Oct 31 08:10:40.768255 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:40.768217    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:42.517860 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.517828    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"]
Oct 31 08:10:42.521152 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.521135    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.557834 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.557809    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.557949 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.557842    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.557949 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.557866    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kvxr\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-kube-api-access-4kvxr\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.558025 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.557950    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.558069 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.558055    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.558118 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.558106    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.558175 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.558157    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.558223 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.558213    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.558259 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.558231    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.575485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.574552    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"]
Oct 31 08:10:42.580486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.578445    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"]
Oct 31 08:10:42.580486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.580101    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.585486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.582047    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.620303 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.620267    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-z29g8\""
Oct 31 08:10:42.620431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.620326    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 08:10:42.620431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.620366    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 08:10:42.620431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.620393    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 08:10:42.623070 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.623047    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"]
Oct 31 08:10:42.659265 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659229    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.659392 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659279    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765fe623-41f8-4f3c-b44e-72293b72fa75-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.659392 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659316    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.659392 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659339    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.659392 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659387    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.659618 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659427    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.659618 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659451    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.659618 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659499    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95xpv\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-kube-api-access-95xpv\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.659618 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659530    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.659618 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659588    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659620    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c694012-1581-4ea4-aee0-35a6a108e0e9-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659650    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659673    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659696    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659726    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659753    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659777    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.659824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659804    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659826    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxzq\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-kube-api-access-rbxzq\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659856    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659883    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659906    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659936    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kvxr\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-kube-api-access-4kvxr\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659959    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.659990    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.660022    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.660100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.660046    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.660388 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.660279    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.660422 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.660392    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.663623 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.660636    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.663623 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.660842    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.663623 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.661261    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.687872 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.673837    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"]
Oct 31 08:10:42.687872 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.677866    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.697107 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.697071    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.719044 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.719011    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.719543 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.719521    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kvxr\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-kube-api-access-4kvxr\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.730019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.729994    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"]
Oct 31 08:10:42.760413 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760391    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.760544 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760421    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765fe623-41f8-4f3c-b44e-72293b72fa75-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.760544 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760442    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.760544 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760487    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.760544 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760529    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95xpv\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-kube-api-access-95xpv\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.760756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760553    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.760756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760602    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.760756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760633    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c694012-1581-4ea4-aee0-35a6a108e0e9-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.760756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760664    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.760756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760688    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.760756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760713    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.760756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760733    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxzq\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-kube-api-access-rbxzq\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.761113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760760    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.761113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760788    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.761113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760810    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.761113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760835    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.761113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760872    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.761113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760899    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.789541 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.769043    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.789541 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.777910    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.789541 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.778149    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.791482 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.760832    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.791482 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.791257    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765fe623-41f8-4f3c-b44e-72293b72fa75-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.791482 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.791339    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.791763 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.791737    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.792010 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.791987    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.792154 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.792131    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.792209 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.792135    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c694012-1581-4ea4-aee0-35a6a108e0e9-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.800439 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.799351    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.800439 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.799651    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.800439 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.799912    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.807360 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.801776    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.835317 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.835297    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:42.846485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.846445    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.846598 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.846511    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95xpv\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-kube-api-access-95xpv\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:42.846598 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.846531    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.847046 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.847026    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxzq\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-kube-api-access-rbxzq\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-85kml\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.901822 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.901784    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:42.913370 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:42.913340    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:43.284202 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.284081    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"]
Oct 31 08:10:43.326580 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:10:43.326552    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9377b9c_f8a1_454a_bda9_69a6b53eef97.slice/crio-e57b613ae4a906ad2f6930332bbc3a4d18903176a27d412445110174470fbebb WatchSource:0}: Error finding container e57b613ae4a906ad2f6930332bbc3a4d18903176a27d412445110174470fbebb: Status 404 returned error can't find the container with id e57b613ae4a906ad2f6930332bbc3a4d18903176a27d412445110174470fbebb
Oct 31 08:10:43.328744 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.328718    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.328826 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.328784    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.328826 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.328813    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.339778 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.339755    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"]
Oct 31 08:10:43.340766 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.340688    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.340943 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.340915    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.341056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.340976    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.390614 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.388952    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"]
Oct 31 08:10:43.396816 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:10:43.396785    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c694012_1581_4ea4_aee0_35a6a108e0e9.slice/crio-5aa4430c229d176eb69cc5e24f95d5ade135cd433858ce273d1abbd419080cef WatchSource:0}: Error finding container 5aa4430c229d176eb69cc5e24f95d5ade135cd433858ce273d1abbd419080cef: Status 404 returned error can't find the container with id 5aa4430c229d176eb69cc5e24f95d5ade135cd433858ce273d1abbd419080cef
Oct 31 08:10:43.399363 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.399333    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.399427 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.399410    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.399485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.399453    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:43.826203 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.826169    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" event={"ID":"c9377b9c-f8a1-454a-bda9-69a6b53eef97","Type":"ContainerStarted","Data":"77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec"}
Oct 31 08:10:43.826717 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.826232    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" event={"ID":"c9377b9c-f8a1-454a-bda9-69a6b53eef97","Type":"ContainerStarted","Data":"e57b613ae4a906ad2f6930332bbc3a4d18903176a27d412445110174470fbebb"}
Oct 31 08:10:43.828475 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.828436    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" event={"ID":"2c694012-1581-4ea4-aee0-35a6a108e0e9","Type":"ContainerStarted","Data":"4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872"}
Oct 31 08:10:43.828565 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.828498    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" event={"ID":"2c694012-1581-4ea4-aee0-35a6a108e0e9","Type":"ContainerStarted","Data":"5aa4430c229d176eb69cc5e24f95d5ade135cd433858ce273d1abbd419080cef"}
Oct 31 08:10:43.830596 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.830569    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" event={"ID":"765fe623-41f8-4f3c-b44e-72293b72fa75","Type":"ContainerStarted","Data":"37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1"}
Oct 31 08:10:43.830693 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.830616    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" event={"ID":"765fe623-41f8-4f3c-b44e-72293b72fa75","Type":"ContainerStarted","Data":"59ea488abfa4250415df26a6273aad33bf8f50f3ead0f786e4399f54f7312092"}
Oct 31 08:10:43.835909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.835891    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:43.906670 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.906586    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:43.918654 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:43.918620    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:44.079545 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.078594    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podStartSLOduration=2.078572928 podStartE2EDuration="2.078572928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:10:43.954677912 +0000 UTC m=+2494.456831914" watchObservedRunningTime="2026-10-31 08:10:44.078572928 +0000 UTC m=+2494.580726719"
Oct 31 08:10:44.198929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.198864    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podStartSLOduration=2.198842643 podStartE2EDuration="2.198842643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:10:44.080291186 +0000 UTC m=+2494.582444988" watchObservedRunningTime="2026-10-31 08:10:44.198842643 +0000 UTC m=+2494.700996434"
Oct 31 08:10:44.850411 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.850368    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 08:10:44.851436 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.851235    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 08:10:44.905111 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.905075    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-85kml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 08:10:44.905817 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.905783    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 08:10:44.919947 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.919915    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 08:10:44.920227 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:44.920195    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 08:10:45.839489 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:45.839195    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 08:10:45.839489 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:45.839270    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 08:10:45.904616 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:45.904582    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-85kml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 08:10:45.905188 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:45.905159    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 08:10:45.916842 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:45.916817    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 08:10:45.917035 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:45.917013    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 08:10:46.839486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:46.838847    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 08:10:46.839486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:46.838910    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 08:10:46.904851 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:46.904815    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-85kml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 08:10:46.905404 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:46.905373    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 08:10:46.916272 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:46.916225    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 08:10:46.916458 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:46.916433    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 08:10:47.837958 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:47.837921    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 08:10:47.838264 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:47.838218    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 08:10:47.919948 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:47.919908    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-85kml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 08:10:47.969506 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:47.921801    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 08:10:47.969506 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:47.969068    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 08:10:47.969777 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:47.969141    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 08:10:48.776577 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.776535    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:48.827880 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.827838    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:48.839955 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.839912    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 08:10:48.840142 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.839974    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 08:10:48.903914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.903826    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-85kml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 08:10:48.904164 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.904135    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 08:10:48.918992 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.918963    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 08:10:48.926553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.926521    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 08:10:48.942665 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.942611    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podStartSLOduration=6.942592047 podStartE2EDuration="6.942592047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:10:44.200803185 +0000 UTC m=+2494.702956968" watchObservedRunningTime="2026-10-31 08:10:48.942592047 +0000 UTC m=+2499.444745838"
Oct 31 08:10:48.946048 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:48.946026    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:10:49.836840 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:49.836804    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 08:10:49.837106 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:49.837079    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 08:10:49.909488 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:49.908881    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-85kml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 08:10:49.909488 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:49.908944    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 08:10:49.917488 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:49.916815    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 08:10:49.917488 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:49.916865    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 08:10:50.839831 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.839794    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:50.840309 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.840290    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:50.841056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.841032    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:10:50.907054 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.907022    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:50.907437 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.907411    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:50.908239 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.908219    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:10:50.918865 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.918843    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:50.919156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.919132    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:50.919893 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:50.919866    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:10:52.939604 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:52.939577    4532 generic.go:356] "Generic (PLEG): container finished" podID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerID="886ae0e6c71be9b1ca76b3494093fa25a2ea985dc71b13f0737c3f2ae9409b65" exitCode=0
Oct 31 08:10:52.940028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:52.939657    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" event={"ID":"08975cf2-7d59-4040-ac89-9fc6e44b54c3","Type":"ContainerDied","Data":"886ae0e6c71be9b1ca76b3494093fa25a2ea985dc71b13f0737c3f2ae9409b65"}
Oct 31 08:10:52.940028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:52.939970    4532 scope.go:122] "RemoveContainer" containerID="886ae0e6c71be9b1ca76b3494093fa25a2ea985dc71b13f0737c3f2ae9409b65"
Oct 31 08:10:52.940971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:52.940944    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:52.941068 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:52.941025    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:52.941133 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:52.941083    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:10:53.945320 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:53.945287    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" event={"ID":"08975cf2-7d59-4040-ac89-9fc6e44b54c3","Type":"ContainerStarted","Data":"a39d90d57b3e12b266b3b8143886ff2b3e2f2ce5f7c5bd41ec9ac401b8f80234"}
Oct 31 08:10:54.621868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:54.621830    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:54.622712 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:54.622677    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 08:10:55.621553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:55.621515    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 08:10:56.622157 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:56.622119    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 08:10:57.621555 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:57.621512    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 08:10:57.755966 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:57.755934    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:58.621608 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:58.621568    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 08:10:59.625337 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:59.625309    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:10:59.626237 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:10:59.626219    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:11:09.310113 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:09.310069    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:11:16.244385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.244352    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"]
Oct 31 08:11:16.253744 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:16.253703    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0" cmd=["/bin/sleep","15"]
Oct 31 08:11:16.253861 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.253761    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:11:16.253861 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:16.253773    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerName="main" containerID="cri-o://4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0"
Oct 31 08:11:16.253861 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.253819    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerName="main" containerID="cri-o://4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0" gracePeriod=60
Oct 31 08:11:16.399414 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:16.399372    4532 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa17e96d_7f23_4556_96f6_79ee2db47fa7.slice/crio-4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 08:11:16.597448 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.597426    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:11:16.699821 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.699793    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-model-cache\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.699952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.699824    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-dshm\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.699952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.699847    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kserve-provision-location\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.699952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.699901    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa17e96d-7f23-4556-96f6-79ee2db47fa7-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fa17e96d-7f23-4556-96f6-79ee2db47fa7-cabundle-cert\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.699952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.699942    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-home\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-home\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.700164 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.699964    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tmp-dir\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.700164 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700088    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-model-cache" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:16.700164 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700112    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kube-api-access-47f8c\" (UniqueName: \"kubernetes.io/projected/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kube-api-access-47f8c\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.700324 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700164    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tls-certs\") pod \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\" (UID: \"fa17e96d-7f23-4556-96f6-79ee2db47fa7\") "
Oct 31 08:11:16.700324 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700207    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tmp-dir" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:16.700324 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700224    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-home" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:16.700501 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700360    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa17e96d-7f23-4556-96f6-79ee2db47fa7-cabundle-cert" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:11:16.700563 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700548    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fa17e96d-7f23-4556-96f6-79ee2db47fa7-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:16.700615 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700569    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:16.700615 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700585    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:16.700615 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.700598    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:16.702158 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.702138    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kube-api-access-47f8c" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "kube-api-access-47f8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:11:16.702235 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.702209    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-dshm" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:16.702558 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.702533    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tls-certs" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:11:16.721690 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.721662    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kserve-provision-location" pod "fa17e96d-7f23-4556-96f6-79ee2db47fa7" (UID: "fa17e96d-7f23-4556-96f6-79ee2db47fa7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:16.801397 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.801371    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47f8c\" (UniqueName: \"kubernetes.io/projected/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kube-api-access-47f8c\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:16.801397 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.801396    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa17e96d-7f23-4556-96f6-79ee2db47fa7-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:16.801574 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.801407    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:16.801574 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:16.801416    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fa17e96d-7f23-4556-96f6-79ee2db47fa7-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:17.044165 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.044136    4532 generic.go:356] "Generic (PLEG): container finished" podID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerID="4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0" exitCode=0
Oct 31 08:11:17.044314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.044187    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"
Oct 31 08:11:17.044314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.044230    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" event={"ID":"fa17e96d-7f23-4556-96f6-79ee2db47fa7","Type":"ContainerDied","Data":"4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0"}
Oct 31 08:11:17.044314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.044285    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz" event={"ID":"fa17e96d-7f23-4556-96f6-79ee2db47fa7","Type":"ContainerDied","Data":"62ce7e691ecb79d4f433443d06cc4ca81dbd2bd000761e27d0cf8bae400d0a6a"}
Oct 31 08:11:17.044314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.044308    4532 scope.go:122] "RemoveContainer" containerID="4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0"
Oct 31 08:11:17.053099 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.053082    4532 scope.go:122] "RemoveContainer" containerID="081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26"
Oct 31 08:11:17.060366 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.060350    4532 scope.go:122] "RemoveContainer" containerID="4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0"
Oct 31 08:11:17.060645 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:17.060625    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0\": container with ID starting with 4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0 not found: ID does not exist" containerID="4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0"
Oct 31 08:11:17.060730 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.060654    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0"} err="failed to get container status \"4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0\": rpc error: code = NotFound desc = could not find container \"4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0\": container with ID starting with 4ac4c57b5b3439e16e570857a9c749b7e91c3148f7eea7e48fafd0a119f007a0 not found: ID does not exist"
Oct 31 08:11:17.060730 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.060679    4532 scope.go:122] "RemoveContainer" containerID="081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26"
Oct 31 08:11:17.060929 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:17.060908    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26\": container with ID starting with 081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26 not found: ID does not exist" containerID="081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26"
Oct 31 08:11:17.060968 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.060937    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26"} err="failed to get container status \"081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26\": rpc error: code = NotFound desc = could not find container \"081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26\": container with ID starting with 081e4497b5172d007b5a824ea16df6b8ec27e9fb21802d791fb56a941cb03f26 not found: ID does not exist"
Oct 31 08:11:17.205859 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.205825    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"]
Oct 31 08:11:17.209435 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.209413    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-9dpbz"]
Oct 31 08:11:17.629902 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:17.629871    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" path="/var/lib/kubelet/pods/fa17e96d-7f23-4556-96f6-79ee2db47fa7/volumes"
Oct 31 08:11:27.703736 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:27.703705    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"]
Oct 31 08:11:27.712150 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:27.712113    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1" cmd=["/bin/sleep","15"]
Oct 31 08:11:27.712277 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:27.712166    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:11:27.712277 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:27.712179    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" podUID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" containerName="main" containerID="cri-o://eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1"
Oct 31 08:11:27.712277 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:27.712222    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" podUID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" containerName="main" containerID="cri-o://eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1" gracePeriod=60
Oct 31 08:11:27.918845 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:27.918789    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" podUID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/health\": dial tcp 10.133.0.38:8000: connect: connection refused"
Oct 31 08:11:28.054639 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.054616    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:11:28.088120 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.088092    4532 generic.go:356] "Generic (PLEG): container finished" podID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" containerID="eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1" exitCode=0
Oct 31 08:11:28.088245 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.088137    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"
Oct 31 08:11:28.088245 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.088183    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" event={"ID":"a897a063-3ddc-4771-9399-fb7bf7ad8a3d","Type":"ContainerDied","Data":"eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1"}
Oct 31 08:11:28.088245 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.088238    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj" event={"ID":"a897a063-3ddc-4771-9399-fb7bf7ad8a3d","Type":"ContainerDied","Data":"91290d6def1edaadf52a92265b67171c3b26012ee71316397278984a29c1973e"}
Oct 31 08:11:28.088388 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.088267    4532 scope.go:122] "RemoveContainer" containerID="eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1"
Oct 31 08:11:28.096122 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.096105    4532 scope.go:122] "RemoveContainer" containerID="eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1"
Oct 31 08:11:28.096376 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:28.096359    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1\": container with ID starting with eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1 not found: ID does not exist" containerID="eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1"
Oct 31 08:11:28.096433 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.096384    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1"} err="failed to get container status \"eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1\": rpc error: code = NotFound desc = could not find container \"eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1\": container with ID starting with eda72e60f5c9abc46f7f0b87ed5120e2d2158f27f80bfcddf8dfdb67c28568e1 not found: ID does not exist"
Oct 31 08:11:28.189236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189212    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-model-cache\") pod \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") "
Oct 31 08:11:28.189384 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189274    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tls-certs\" (UniqueName: \"kubernetes.io/secret/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tls-certs\") pod \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") "
Oct 31 08:11:28.189384 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189320    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-dshm\") pod \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") "
Oct 31 08:11:28.189384 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189342    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-home\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-home\") pod \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") "
Oct 31 08:11:28.189384 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189374    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tmp-dir\") pod \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") "
Oct 31 08:11:28.189610 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189405    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-kube-api-access-ncr8v\" (UniqueName: \"kubernetes.io/projected/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-kube-api-access-ncr8v\") pod \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\" (UID: \"a897a063-3ddc-4771-9399-fb7bf7ad8a3d\") "
Oct 31 08:11:28.189610 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189538    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-model-cache" pod "a897a063-3ddc-4771-9399-fb7bf7ad8a3d" (UID: "a897a063-3ddc-4771-9399-fb7bf7ad8a3d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:28.189818 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189766    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tmp-dir" pod "a897a063-3ddc-4771-9399-fb7bf7ad8a3d" (UID: "a897a063-3ddc-4771-9399-fb7bf7ad8a3d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:28.189925 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189789    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:28.189925 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.189841    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-home" pod "a897a063-3ddc-4771-9399-fb7bf7ad8a3d" (UID: "a897a063-3ddc-4771-9399-fb7bf7ad8a3d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:28.191648 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.191617    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-kube-api-access-ncr8v" pod "a897a063-3ddc-4771-9399-fb7bf7ad8a3d" (UID: "a897a063-3ddc-4771-9399-fb7bf7ad8a3d"). InnerVolumeSpecName "kube-api-access-ncr8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:11:28.191732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.191624    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tls-certs" pod "a897a063-3ddc-4771-9399-fb7bf7ad8a3d" (UID: "a897a063-3ddc-4771-9399-fb7bf7ad8a3d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:11:28.191852 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.191830    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-dshm" pod "a897a063-3ddc-4771-9399-fb7bf7ad8a3d" (UID: "a897a063-3ddc-4771-9399-fb7bf7ad8a3d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:28.290587 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.290566    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:28.290587 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.290586    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:28.290720 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.290595    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:28.290720 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.290621    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:28.290720 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.290631    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncr8v\" (UniqueName: \"kubernetes.io/projected/a897a063-3ddc-4771-9399-fb7bf7ad8a3d-kube-api-access-ncr8v\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:28.507021 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.506992    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"]
Oct 31 08:11:28.509147 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:28.509120    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbv6zrj"]
Oct 31 08:11:29.624607 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:29.624577    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" path="/var/lib/kubelet/pods/a897a063-3ddc-4771-9399-fb7bf7ad8a3d/volumes"
Oct 31 08:11:36.979006 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:36.978975    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"]
Oct 31 08:11:36.979416 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:36.979194    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy" containerID="cri-o://a39d90d57b3e12b266b3b8143886ff2b3e2f2ce5f7c5bd41ec9ac401b8f80234" gracePeriod=30
Oct 31 08:11:39.311235 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:11:39.311194    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:11:42.146977 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.146946    4532 generic.go:356] "Generic (PLEG): container finished" podID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerID="a39d90d57b3e12b266b3b8143886ff2b3e2f2ce5f7c5bd41ec9ac401b8f80234" exitCode=0
Oct 31 08:11:42.147400 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.146994    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" event={"ID":"08975cf2-7d59-4040-ac89-9fc6e44b54c3","Type":"ContainerDied","Data":"a39d90d57b3e12b266b3b8143886ff2b3e2f2ce5f7c5bd41ec9ac401b8f80234"}
Oct 31 08:11:42.147400 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.147048    4532 scope.go:122] "RemoveContainer" containerID="886ae0e6c71be9b1ca76b3494093fa25a2ea985dc71b13f0737c3f2ae9409b65"
Oct 31 08:11:42.214440 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.214412    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:11:42.298663 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298642    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-envoy\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.298803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298695    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-podinfo\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.298803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298721    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-data\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.298803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298741    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-kube-api-access-mbvwf\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-kube-api-access-mbvwf\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.298803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298758    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-token\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-token\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.298803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298795    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-credential-socket\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.299156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298823    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-certs\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.299156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298889    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-socket\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.299156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298915    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istiod-ca-cert\") pod \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\" (UID: \"08975cf2-7d59-4040-ac89-9fc6e44b54c3\") "
Oct 31 08:11:42.299156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.298994    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-data" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:42.299156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.299115    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-certs" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:42.299425 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.299151    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-socket" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:42.299425 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.299210    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.299425 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.299230    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.299425 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.299261    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-credential-socket" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:42.299425 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.299336    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istiod-ca-cert" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:11:42.301054 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.301027    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-token" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:11:42.301235 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.301218    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-podinfo" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:11:42.301300 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.301240    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-envoy" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:11:42.301353 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.301320    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-kube-api-access-mbvwf" pod "08975cf2-7d59-4040-ac89-9fc6e44b54c3" (UID: "08975cf2-7d59-4040-ac89-9fc6e44b54c3"). InnerVolumeSpecName "kube-api-access-mbvwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:11:42.400720 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.400678    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.400720 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.400708    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.400905 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.400724    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbvwf\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-kube-api-access-mbvwf\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.400905 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.400739    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.400905 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.400754    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.400905 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.400767    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08975cf2-7d59-4040-ac89-9fc6e44b54c3-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:42.400905 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:42.400782    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08975cf2-7d59-4040-ac89-9fc6e44b54c3-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:11:43.152092 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:43.152058    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"
Oct 31 08:11:43.152538 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:43.152094    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2" event={"ID":"08975cf2-7d59-4040-ac89-9fc6e44b54c3","Type":"ContainerDied","Data":"83eca5447641e2ee0756db02cea68725c15f115dad55d4bb7d815e110523a8cd"}
Oct 31 08:11:43.152538 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:43.152147    4532 scope.go:122] "RemoveContainer" containerID="a39d90d57b3e12b266b3b8143886ff2b3e2f2ce5f7c5bd41ec9ac401b8f80234"
Oct 31 08:11:43.362800 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:43.362768    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"]
Oct 31 08:11:43.412437 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:43.412374    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/router-gateway-1-openshift-default-764c68896-g7tm2"]
Oct 31 08:11:43.624319 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:11:43.624288    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" path="/var/lib/kubelet/pods/08975cf2-7d59-4040-ac89-9fc6e44b54c3/volumes"
Oct 31 08:12:09.310641 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:12:09.310605    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:12:09.483373 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483340    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"]
Oct 31 08:12:09.483682 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483670    4532 cpu_manager.go:414] "containerMap: removing container" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483684    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483694    4532 cpu_manager.go:414] "containerMap: removing container" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483699    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483708    4532 cpu_manager.go:414] "containerMap: removing container" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerName="storage-initializer"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483713    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerName="storage-initializer"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483720    4532 cpu_manager.go:414] "containerMap: removing container" podUID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" containerName="main"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483724    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" containerName="main"
Oct 31 08:12:09.483732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483732    4532 cpu_manager.go:414] "containerMap: removing container" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerName="main"
Oct 31 08:12:09.483962 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483737    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerName="main"
Oct 31 08:12:09.483962 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483786    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy"
Oct 31 08:12:09.483962 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483793    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="08975cf2-7d59-4040-ac89-9fc6e44b54c3" containerName="istio-proxy"
Oct 31 08:12:09.483962 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483800    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="a897a063-3ddc-4771-9399-fb7bf7ad8a3d" containerName="main"
Oct 31 08:12:09.483962 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.483806    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa17e96d-7f23-4556-96f6-79ee2db47fa7" containerName="main"
Oct 31 08:12:09.486599 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.486582    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.587871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.587801    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"kube-root-ca.crt\""
Oct 31 08:12:09.587871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.587809    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Oct 31 08:12:09.588033 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.587991    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"openshift-service-ca.crt\""
Oct 31 08:12:09.588115 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.588101    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"default-dockercfg-x6pvm\""
Oct 31 08:12:09.589413 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.589395    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"]
Oct 31 08:12:09.608375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.608347    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.608375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.608378    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.608585 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.608413    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.608585 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.608554    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.608684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.608620    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6mln\" (UniqueName: \"kubernetes.io/projected/aaeeca87-63c2-48bb-95f7-61054ab13d06-kube-api-access-h6mln\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.608719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.608682    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaeeca87-63c2-48bb-95f7-61054ab13d06-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.709644 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.709614    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaeeca87-63c2-48bb-95f7-61054ab13d06-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.709839 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.709654    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.709839 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.709676    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.709982 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.709864    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.709982 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.709901    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.709982 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.709967    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6mln\" (UniqueName: \"kubernetes.io/projected/aaeeca87-63c2-48bb-95f7-61054ab13d06-kube-api-access-h6mln\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.710147 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.710091    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-home\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.710260 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.710238    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-tmp-dir\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.710316 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.710302    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-model-cache\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.711802 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.711784    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-dshm\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.760441 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.760414    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Oct 31 08:12:09.772108 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.772084    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaeeca87-63c2-48bb-95f7-61054ab13d06-tls-certs\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:09.813970 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.813949    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"kube-root-ca.crt\""
Oct 31 08:12:09.874626 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.874579    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"openshift-service-ca.crt\""
Oct 31 08:12:09.886270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:09.886246    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6mln\" (UniqueName: \"kubernetes.io/projected/aaeeca87-63c2-48bb-95f7-61054ab13d06-kube-api-access-h6mln\") pod \"llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:10.149102 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:10.149021    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"default-dockercfg-x6pvm\""
Oct 31 08:12:10.158820 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:10.158796    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:10.528815 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:10.528755    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"]
Oct 31 08:12:10.531349 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:12:10.531325    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaeeca87_63c2_48bb_95f7_61054ab13d06.slice/crio-2d8d4b92c681f625405f928a8fdc9eff8f048e4687a1b7004ae7c0777fdba1cd WatchSource:0}: Error finding container 2d8d4b92c681f625405f928a8fdc9eff8f048e4687a1b7004ae7c0777fdba1cd: Status 404 returned error can't find the container with id 2d8d4b92c681f625405f928a8fdc9eff8f048e4687a1b7004ae7c0777fdba1cd
Oct 31 08:12:11.270173 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:11.270139    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" event={"ID":"aaeeca87-63c2-48bb-95f7-61054ab13d06","Type":"ContainerStarted","Data":"487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c"}
Oct 31 08:12:11.270335 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:11.270190    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" event={"ID":"aaeeca87-63c2-48bb-95f7-61054ab13d06","Type":"ContainerStarted","Data":"2d8d4b92c681f625405f928a8fdc9eff8f048e4687a1b7004ae7c0777fdba1cd"}
Oct 31 08:12:11.383591 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:11.383542    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" podStartSLOduration=2.383524849 podStartE2EDuration="2.383524849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:12:11.382516056 +0000 UTC m=+2581.884669847" watchObservedRunningTime="2026-10-31 08:12:11.383524849 +0000 UTC m=+2581.885678642"
Oct 31 08:12:12.159959 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:12.159927    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:20.159212 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:20.159173    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:20.172210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:20.172186    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:20.321673 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:20.321646    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:39.310834 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:12:39.310794    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:12:49.249170 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.249136    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"]
Oct 31 08:12:49.258412 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:12:49.258368    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c" cmd=["/bin/sleep","15"]
Oct 31 08:12:49.258566 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.258442    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:12:49.258566 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:12:49.258459    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" podUID="aaeeca87-63c2-48bb-95f7-61054ab13d06" containerName="main" containerID="cri-o://487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c"
Oct 31 08:12:49.258566 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.258544    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" podUID="aaeeca87-63c2-48bb-95f7-61054ab13d06" containerName="main" containerID="cri-o://487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c" gracePeriod=60
Oct 31 08:12:49.311261 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.311226    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" podUID="aaeeca87-63c2-48bb-95f7-61054ab13d06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": dial tcp 10.133.0.42:8000: connect: connection refused"
Oct 31 08:12:49.596831 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.596808    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:49.643013 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.642983    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-model-cache\") pod \"aaeeca87-63c2-48bb-95f7-61054ab13d06\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") "
Oct 31 08:12:49.643145 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643045    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaeeca87-63c2-48bb-95f7-61054ab13d06-tls-certs\" (UniqueName: \"kubernetes.io/secret/aaeeca87-63c2-48bb-95f7-61054ab13d06-tls-certs\") pod \"aaeeca87-63c2-48bb-95f7-61054ab13d06\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") "
Oct 31 08:12:49.643145 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643086    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-dshm\") pod \"aaeeca87-63c2-48bb-95f7-61054ab13d06\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") "
Oct 31 08:12:49.643263 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643146    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaeeca87-63c2-48bb-95f7-61054ab13d06-kube-api-access-h6mln\" (UniqueName: \"kubernetes.io/projected/aaeeca87-63c2-48bb-95f7-61054ab13d06-kube-api-access-h6mln\") pod \"aaeeca87-63c2-48bb-95f7-61054ab13d06\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") "
Oct 31 08:12:49.643263 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643213    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-tmp-dir\") pod \"aaeeca87-63c2-48bb-95f7-61054ab13d06\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") "
Oct 31 08:12:49.643263 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643247    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-home\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-home\") pod \"aaeeca87-63c2-48bb-95f7-61054ab13d06\" (UID: \"aaeeca87-63c2-48bb-95f7-61054ab13d06\") "
Oct 31 08:12:49.643413 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643261    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-model-cache" pod "aaeeca87-63c2-48bb-95f7-61054ab13d06" (UID: "aaeeca87-63c2-48bb-95f7-61054ab13d06"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:12:49.643533 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643517    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:12:49.643592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643527    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-tmp-dir" pod "aaeeca87-63c2-48bb-95f7-61054ab13d06" (UID: "aaeeca87-63c2-48bb-95f7-61054ab13d06"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:12:49.643643 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.643615    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-home" pod "aaeeca87-63c2-48bb-95f7-61054ab13d06" (UID: "aaeeca87-63c2-48bb-95f7-61054ab13d06"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:12:49.645213 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.645187    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-dshm" pod "aaeeca87-63c2-48bb-95f7-61054ab13d06" (UID: "aaeeca87-63c2-48bb-95f7-61054ab13d06"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:12:49.645213 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.645204    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaeeca87-63c2-48bb-95f7-61054ab13d06-tls-certs" pod "aaeeca87-63c2-48bb-95f7-61054ab13d06" (UID: "aaeeca87-63c2-48bb-95f7-61054ab13d06"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:12:49.645347 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.645243    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaeeca87-63c2-48bb-95f7-61054ab13d06-kube-api-access-h6mln" pod "aaeeca87-63c2-48bb-95f7-61054ab13d06" (UID: "aaeeca87-63c2-48bb-95f7-61054ab13d06"). InnerVolumeSpecName "kube-api-access-h6mln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:12:49.744309 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.744283    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaeeca87-63c2-48bb-95f7-61054ab13d06-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:12:49.744309 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.744309    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:12:49.744491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.744323    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6mln\" (UniqueName: \"kubernetes.io/projected/aaeeca87-63c2-48bb-95f7-61054ab13d06-kube-api-access-h6mln\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:12:49.744491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.744335    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:12:49.744491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:49.744347    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaeeca87-63c2-48bb-95f7-61054ab13d06-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:12:50.431103 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.431067    4532 generic.go:356] "Generic (PLEG): container finished" podID="aaeeca87-63c2-48bb-95f7-61054ab13d06" containerID="487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c" exitCode=0
Oct 31 08:12:50.431537 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.431123    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"
Oct 31 08:12:50.431537 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.431161    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" event={"ID":"aaeeca87-63c2-48bb-95f7-61054ab13d06","Type":"ContainerDied","Data":"487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c"}
Oct 31 08:12:50.431537 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.431213    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq" event={"ID":"aaeeca87-63c2-48bb-95f7-61054ab13d06","Type":"ContainerDied","Data":"2d8d4b92c681f625405f928a8fdc9eff8f048e4687a1b7004ae7c0777fdba1cd"}
Oct 31 08:12:50.431537 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.431242    4532 scope.go:122] "RemoveContainer" containerID="487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c"
Oct 31 08:12:50.439783 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.439765    4532 scope.go:122] "RemoveContainer" containerID="487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c"
Oct 31 08:12:50.440055 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:12:50.440038    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c\": container with ID starting with 487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c not found: ID does not exist" containerID="487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c"
Oct 31 08:12:50.440099 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.440062    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c"} err="failed to get container status \"487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c\": rpc error: code = NotFound desc = could not find container \"487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c\": container with ID starting with 487fe1e1afa2f849fde74798f8e770a8b73bd593cf80f25bf57a94db36559f9c not found: ID does not exist"
Oct 31 08:12:50.590882 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.590857    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"]
Oct 31 08:12:50.639754 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:50.639726    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc-router-managed-test-llm-e45d1f79-kserve-58fbd56d5-sc8qq"]
Oct 31 08:12:51.623769 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:12:51.623738    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaeeca87-63c2-48bb-95f7-61054ab13d06" path="/var/lib/kubelet/pods/aaeeca87-63c2-48bb-95f7-61054ab13d06/volumes"
Oct 31 08:13:09.311083 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:13:09.311046    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:13:39.313501 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:13:39.313452    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:13:44.449028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.448991    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"]
Oct 31 08:13:44.449522 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.449316    4532 cpu_manager.go:414] "containerMap: removing container" podUID="aaeeca87-63c2-48bb-95f7-61054ab13d06" containerName="main"
Oct 31 08:13:44.449522 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.449326    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aaeeca87-63c2-48bb-95f7-61054ab13d06" containerName="main"
Oct 31 08:13:44.449522 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.449374    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="aaeeca87-63c2-48bb-95f7-61054ab13d06" containerName="main"
Oct 31 08:13:44.452252 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.452236    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.503885 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.503859    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Oct 31 08:13:44.553158 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.553134    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Oct 31 08:13:44.553328 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.553310    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-p52xl\""
Oct 31 08:13:44.553410 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.553398    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Oct 31 08:13:44.556446 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.555894    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"]
Oct 31 08:13:44.565765 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.565740    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.565890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.565774    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.565890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.565798    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.566011 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.565938    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb6sz\" (UniqueName: \"kubernetes.io/projected/d6946536-c280-45ff-8da4-521e9b486064-kube-api-access-xb6sz\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.566057 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.566019    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6946536-c280-45ff-8da4-521e9b486064-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.566057 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.566040    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.666809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.666782    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.666952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.666818    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.666952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.666836    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.666952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.666865    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb6sz\" (UniqueName: \"kubernetes.io/projected/d6946536-c280-45ff-8da4-521e9b486064-kube-api-access-xb6sz\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.666952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.666919    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6946536-c280-45ff-8da4-521e9b486064-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.666952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.666946    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.667279 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.667254    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.667279 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.667271    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.667403 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.667332    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.669222 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.669196    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.669562 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.669545    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6946536-c280-45ff-8da4-521e9b486064-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.722555 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.722505    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb6sz\" (UniqueName: \"kubernetes.io/projected/d6946536-c280-45ff-8da4-521e9b486064-kube-api-access-xb6sz\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:44.764081 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:44.764057    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:45.134736 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:45.134710    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"]
Oct 31 08:13:45.137265 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:13:45.137240    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6946536_c280_45ff_8da4_521e9b486064.slice/crio-f05b7ec6be3425a4363a89cee4257a74f7c9576f703a3dccd44da1a247131962 WatchSource:0}: Error finding container f05b7ec6be3425a4363a89cee4257a74f7c9576f703a3dccd44da1a247131962: Status 404 returned error can't find the container with id f05b7ec6be3425a4363a89cee4257a74f7c9576f703a3dccd44da1a247131962
Oct 31 08:13:45.641861 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:45.641827    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" event={"ID":"d6946536-c280-45ff-8da4-521e9b486064","Type":"ContainerStarted","Data":"88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd"}
Oct 31 08:13:45.642236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:45.641874    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" event={"ID":"d6946536-c280-45ff-8da4-521e9b486064","Type":"ContainerStarted","Data":"f05b7ec6be3425a4363a89cee4257a74f7c9576f703a3dccd44da1a247131962"}
Oct 31 08:13:45.754941 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:45.754891    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" podStartSLOduration=1.75487875 podStartE2EDuration="1.75487875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:13:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:13:45.753208455 +0000 UTC m=+2676.255362244" watchObservedRunningTime="2026-10-31 08:13:45.75487875 +0000 UTC m=+2676.257032539"
Oct 31 08:13:45.764583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:45.764557    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:54.764974 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:54.764940    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:54.777472 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:54.777438    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:13:55.693103 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:13:55.693074    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:14:09.311002 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:14:09.310965    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:14:39.312091 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:14:39.312055    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:15:09.310706 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:15:09.310670    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:15:39.310968 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:15:39.310931    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:16:09.310659 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:16:09.310622    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:16:11.844075 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:11.844039    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"]
Oct 31 08:16:11.861211 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:16:11.861170    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd" cmd=["/bin/sleep","15"]
Oct 31 08:16:11.861386 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:11.861231    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:16:11.861386 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:16:11.861244    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" podUID="d6946536-c280-45ff-8da4-521e9b486064" containerName="main" containerID="cri-o://88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd"
Oct 31 08:16:11.861386 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:11.861291    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" podUID="d6946536-c280-45ff-8da4-521e9b486064" containerName="main" containerID="cri-o://88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd" gracePeriod=60
Oct 31 08:16:12.205940 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.205912    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:16:12.213008 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.212980    4532 generic.go:356] "Generic (PLEG): container finished" podID="d6946536-c280-45ff-8da4-521e9b486064" containerID="88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd" exitCode=0
Oct 31 08:16:12.213146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.213018    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"
Oct 31 08:16:12.213146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.213050    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" event={"ID":"d6946536-c280-45ff-8da4-521e9b486064","Type":"ContainerDied","Data":"88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd"}
Oct 31 08:16:12.213146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.213102    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c" event={"ID":"d6946536-c280-45ff-8da4-521e9b486064","Type":"ContainerDied","Data":"f05b7ec6be3425a4363a89cee4257a74f7c9576f703a3dccd44da1a247131962"}
Oct 31 08:16:12.213146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.213130    4532 scope.go:122] "RemoveContainer" containerID="88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd"
Oct 31 08:16:12.220802 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.220784    4532 scope.go:122] "RemoveContainer" containerID="88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd"
Oct 31 08:16:12.221085 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:16:12.221063    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd\": container with ID starting with 88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd not found: ID does not exist" containerID="88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd"
Oct 31 08:16:12.221184 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.221089    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd"} err="failed to get container status \"88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd\": rpc error: code = NotFound desc = could not find container \"88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd\": container with ID starting with 88959a9b2d499d998e72cad8bfb88a91d6793e231f01d720a53f102d03dca8dd not found: ID does not exist"
Oct 31 08:16:12.291645 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.291622    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-dshm\") pod \"d6946536-c280-45ff-8da4-521e9b486064\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") "
Oct 31 08:16:12.291758 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.291701    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-model-cache\") pod \"d6946536-c280-45ff-8da4-521e9b486064\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") "
Oct 31 08:16:12.291758 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.291743    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6946536-c280-45ff-8da4-521e9b486064-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6946536-c280-45ff-8da4-521e9b486064-tls-certs\") pod \"d6946536-c280-45ff-8da4-521e9b486064\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") "
Oct 31 08:16:12.291875 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.291774    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-tmp-dir\") pod \"d6946536-c280-45ff-8da4-521e9b486064\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") "
Oct 31 08:16:12.291875 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.291825    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6946536-c280-45ff-8da4-521e9b486064-kube-api-access-xb6sz\" (UniqueName: \"kubernetes.io/projected/d6946536-c280-45ff-8da4-521e9b486064-kube-api-access-xb6sz\") pod \"d6946536-c280-45ff-8da4-521e9b486064\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") "
Oct 31 08:16:12.292037 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.291881    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-home\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-home\") pod \"d6946536-c280-45ff-8da4-521e9b486064\" (UID: \"d6946536-c280-45ff-8da4-521e9b486064\") "
Oct 31 08:16:12.292037 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.292001    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-model-cache" pod "d6946536-c280-45ff-8da4-521e9b486064" (UID: "d6946536-c280-45ff-8da4-521e9b486064"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:16:12.292156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.292130    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-tmp-dir" pod "d6946536-c280-45ff-8da4-521e9b486064" (UID: "d6946536-c280-45ff-8da4-521e9b486064"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:16:12.292211 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.292173    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:16:12.292253 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.292209    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-home" pod "d6946536-c280-45ff-8da4-521e9b486064" (UID: "d6946536-c280-45ff-8da4-521e9b486064"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:16:12.293816 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.293795    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-dshm" pod "d6946536-c280-45ff-8da4-521e9b486064" (UID: "d6946536-c280-45ff-8da4-521e9b486064"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:16:12.293914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.293846    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6946536-c280-45ff-8da4-521e9b486064-tls-certs" pod "d6946536-c280-45ff-8da4-521e9b486064" (UID: "d6946536-c280-45ff-8da4-521e9b486064"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:16:12.293988 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.293969    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6946536-c280-45ff-8da4-521e9b486064-kube-api-access-xb6sz" pod "d6946536-c280-45ff-8da4-521e9b486064" (UID: "d6946536-c280-45ff-8da4-521e9b486064"). InnerVolumeSpecName "kube-api-access-xb6sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:16:12.393119 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.393065    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb6sz\" (UniqueName: \"kubernetes.io/projected/d6946536-c280-45ff-8da4-521e9b486064-kube-api-access-xb6sz\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:16:12.393119 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.393087    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:16:12.393119 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.393097    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:16:12.393119 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.393105    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6946536-c280-45ff-8da4-521e9b486064-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:16:12.393119 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.393114    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6946536-c280-45ff-8da4-521e9b486064-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:16:12.630660 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.630634    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"]
Oct 31 08:16:12.632580 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:12.632557    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848cf47c"]
Oct 31 08:16:13.624521 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:16:13.624489    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6946536-c280-45ff-8da4-521e9b486064" path="/var/lib/kubelet/pods/d6946536-c280-45ff-8da4-521e9b486064/volumes"
Oct 31 08:16:39.312074 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:16:39.312037    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:17:09.310682 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:17:09.310637    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:17:39.311006 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:17:39.310967    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:17:53.015252 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.015177    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"]
Oct 31 08:17:53.015703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.015543    4532 cpu_manager.go:414] "containerMap: removing container" podUID="d6946536-c280-45ff-8da4-521e9b486064" containerName="main"
Oct 31 08:17:53.015703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.015554    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6946536-c280-45ff-8da4-521e9b486064" containerName="main"
Oct 31 08:17:53.015703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.015611    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6946536-c280-45ff-8da4-521e9b486064" containerName="main"
Oct 31 08:17:53.018545 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.018525    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.070410 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.070379    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Oct 31 08:17:53.119669 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.119632    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-4htpt\""
Oct 31 08:17:53.119669 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.119632    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Oct 31 08:17:53.119863 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.119675    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Oct 31 08:17:53.121042 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.121021    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"]
Oct 31 08:17:53.138170 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.138146    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.138277 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.138176    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.138277 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.138220    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.138368 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.138334    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bmvn\" (UniqueName: \"kubernetes.io/projected/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-kube-api-access-2bmvn\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.138422 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.138364    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.138422 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.138382    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.238836 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.238798    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.238836 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.238835    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.239078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.238875    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.239078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.238896    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bmvn\" (UniqueName: \"kubernetes.io/projected/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-kube-api-access-2bmvn\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.239078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.238916    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.239078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.238931    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.239313 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.239290    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.239379 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.239297    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.239423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.239382    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.241199 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.241179    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.241610 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.241588    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.297380 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.297353    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bmvn\" (UniqueName: \"kubernetes.io/projected/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-kube-api-access-2bmvn\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.330881 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.330853    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:17:53.696312 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:53.696234    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"]
Oct 31 08:17:53.699310 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:17:53.699282    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9175ee3e_e5fa_4a57_ac9c_9cdd02622bb1.slice/crio-8e3358b95623eca7b123ab930eeb676f48db2526288892d2c171730e5d575d2a WatchSource:0}: Error finding container 8e3358b95623eca7b123ab930eeb676f48db2526288892d2c171730e5d575d2a: Status 404 returned error can't find the container with id 8e3358b95623eca7b123ab930eeb676f48db2526288892d2c171730e5d575d2a
Oct 31 08:17:54.632299 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:54.632262    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" event={"ID":"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1","Type":"ContainerStarted","Data":"bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570"}
Oct 31 08:17:54.632800 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:54.632322    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" event={"ID":"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1","Type":"ContainerStarted","Data":"8e3358b95623eca7b123ab930eeb676f48db2526288892d2c171730e5d575d2a"}
Oct 31 08:17:54.745017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:54.744969    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" podStartSLOduration=2.74495479 podStartE2EDuration="2.74495479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:17:54.744454372 +0000 UTC m=+2925.246608163" watchObservedRunningTime="2026-10-31 08:17:54.74495479 +0000 UTC m=+2925.247108580"
Oct 31 08:17:55.331795 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:17:55.331763    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:18:03.331612 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:03.331577    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:18:03.344415 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:03.344386    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:18:03.676153 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:03.676082    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:18:09.310506 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:18:09.310455    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:18:32.928676 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:32.928643    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"]
Oct 31 08:18:32.938572 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:18:32.938538    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570" cmd=["/bin/sleep","15"]
Oct 31 08:18:32.938704 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:32.938592    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:18:32.938704 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:18:32.938604    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" podUID="9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" containerName="main" containerID="cri-o://bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570"
Oct 31 08:18:32.938704 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:32.938648    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" podUID="9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" containerName="main" containerID="cri-o://bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570" gracePeriod=60
Oct 31 08:18:33.282181 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.282148    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:18:33.355347 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355322    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-home\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-home\") pod \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") "
Oct 31 08:18:33.355497 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355364    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-model-cache\") pod \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") "
Oct 31 08:18:33.355497 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355390    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tmp-dir\") pod \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") "
Oct 31 08:18:33.355497 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355410    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tls-certs\" (UniqueName: \"kubernetes.io/secret/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tls-certs\") pod \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") "
Oct 31 08:18:33.355497 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355429    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-dshm\") pod \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") "
Oct 31 08:18:33.355497 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355451    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-kube-api-access-2bmvn\" (UniqueName: \"kubernetes.io/projected/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-kube-api-access-2bmvn\") pod \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\" (UID: \"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1\") "
Oct 31 08:18:33.355770 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355639    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-home" pod "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" (UID: "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:18:33.355770 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355650    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tmp-dir" pod "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" (UID: "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:18:33.355881 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.355804    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-model-cache" pod "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" (UID: "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:18:33.357755 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.357720    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tls-certs" pod "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" (UID: "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:18:33.357858 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.357765    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-dshm" pod "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" (UID: "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:18:33.357858 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.357834    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-kube-api-access-2bmvn" pod "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" (UID: "9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1"). InnerVolumeSpecName "kube-api-access-2bmvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:18:33.456431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.456369    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:18:33.456431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.456393    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:18:33.456431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.456402    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:18:33.456431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.456410    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:18:33.456431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.456418    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:18:33.456431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.456427    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bmvn\" (UniqueName: \"kubernetes.io/projected/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1-kube-api-access-2bmvn\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:18:33.784406 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.784375    4532 generic.go:356] "Generic (PLEG): container finished" podID="9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" containerID="bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570" exitCode=0
Oct 31 08:18:33.784561 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.784423    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" event={"ID":"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1","Type":"ContainerDied","Data":"bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570"}
Oct 31 08:18:33.784561 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.784436    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"
Oct 31 08:18:33.784561 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.784482    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf" event={"ID":"9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1","Type":"ContainerDied","Data":"8e3358b95623eca7b123ab930eeb676f48db2526288892d2c171730e5d575d2a"}
Oct 31 08:18:33.784561 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.784504    4532 scope.go:122] "RemoveContainer" containerID="bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570"
Oct 31 08:18:33.792879 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.792860    4532 scope.go:122] "RemoveContainer" containerID="bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570"
Oct 31 08:18:33.793155 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:18:33.793134    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570\": container with ID starting with bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570 not found: ID does not exist" containerID="bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570"
Oct 31 08:18:33.793197 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.793164    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570"} err="failed to get container status \"bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570\": rpc error: code = NotFound desc = could not find container \"bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570\": container with ID starting with bccd2fd26dceb51b06969ce43e276cbe4c37cb4cc283d7c1d97eaaa647283570 not found: ID does not exist"
Oct 31 08:18:33.944955 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.944922    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"]
Oct 31 08:18:33.993446 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:33.993420    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777x9fpf"]
Oct 31 08:18:35.624112 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:18:35.624075    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" path="/var/lib/kubelet/pods/9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1/volumes"
Oct 31 08:18:39.310845 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:18:39.310812    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:19:09.310526 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:19:09.310491    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:19:39.310562 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:19:39.310527    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:20:09.310234 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:20:09.310199    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:20:19.403017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.402983    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"]
Oct 31 08:20:19.403565 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.403337    4532 cpu_manager.go:414] "containerMap: removing container" podUID="9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" containerName="main"
Oct 31 08:20:19.403565 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.403349    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" containerName="main"
Oct 31 08:20:19.403565 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.403407    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="9175ee3e-e5fa-4a57-ac9c-9cdd02622bb1" containerName="main"
Oct 31 08:20:19.406368 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.406349    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.461669 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.461636    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"]
Oct 31 08:20:19.484048 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484019    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.484212 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484067    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96704e56-2c3f-451e-8595-208e419b07b9-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.484212 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484131    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-home\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.484212 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484148    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.484212 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484165    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-dshm\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.484212 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484186    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.484481 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484258    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96704e56-2c3f-451e-8595-208e419b07b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.484481 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.484352    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm4df\" (UniqueName: \"kubernetes.io/projected/96704e56-2c3f-451e-8595-208e419b07b9-kube-api-access-dm4df\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.509357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.509329    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"default-dockercfg-gzt6x\""
Oct 31 08:20:19.509357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.509333    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:20:19.509566 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.509336    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"kube-root-ca.crt\""
Oct 31 08:20:19.509566 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.509346    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"seaweedfs-s3-creds\""
Oct 31 08:20:19.509566 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.509356    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"openshift-service-ca.crt\""
Oct 31 08:20:19.509566 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.509414    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs\""
Oct 31 08:20:19.585198 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585168    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585215    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96704e56-2c3f-451e-8595-208e419b07b9-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585243    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-home\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585261    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585279    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-dshm\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585301    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585327    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96704e56-2c3f-451e-8595-208e419b07b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585374    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm4df\" (UniqueName: \"kubernetes.io/projected/96704e56-2c3f-451e-8595-208e419b07b9-kube-api-access-dm4df\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585689    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585861 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585836    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-home\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585934 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585877    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.585934 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.585918    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.586212 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.586196    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96704e56-2c3f-451e-8595-208e419b07b9-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.588175 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.588154    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-dshm\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.588408 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.588393    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96704e56-2c3f-451e-8595-208e419b07b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.641577 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.641547    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm4df\" (UniqueName: \"kubernetes.io/projected/96704e56-2c3f-451e-8595-208e419b07b9-kube-api-access-dm4df\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:19.722039 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:19.721953    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:20.140798 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:20.140764    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"]
Oct 31 08:20:20.144110 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:20:20.144087    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96704e56_2c3f_451e_8595_208e419b07b9.slice/crio-e5681c277f2b8100c91deb2bf49de1460a3f2ee5aeff315187ae599a457c5161 WatchSource:0}: Error finding container e5681c277f2b8100c91deb2bf49de1460a3f2ee5aeff315187ae599a457c5161: Status 404 returned error can't find the container with id e5681c277f2b8100c91deb2bf49de1460a3f2ee5aeff315187ae599a457c5161
Oct 31 08:20:20.203289 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:20.203258    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" event={"ID":"96704e56-2c3f-451e-8595-208e419b07b9","Type":"ContainerStarted","Data":"e5681c277f2b8100c91deb2bf49de1460a3f2ee5aeff315187ae599a457c5161"}
Oct 31 08:20:21.209726 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:21.209681    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" event={"ID":"96704e56-2c3f-451e-8595-208e419b07b9","Type":"ContainerStarted","Data":"db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b"}
Oct 31 08:20:24.222976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:24.222941    4532 generic.go:356] "Generic (PLEG): container finished" podID="96704e56-2c3f-451e-8595-208e419b07b9" containerID="db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b" exitCode=0
Oct 31 08:20:24.223365 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:24.223015    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" event={"ID":"96704e56-2c3f-451e-8595-208e419b07b9","Type":"ContainerDied","Data":"db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b"}
Oct 31 08:20:24.224339 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:24.224315    4532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 08:20:39.316796 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:20:39.316759    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:20:53.369512 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:53.369453    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" event={"ID":"96704e56-2c3f-451e-8595-208e419b07b9","Type":"ContainerStarted","Data":"2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6"}
Oct 31 08:20:53.584572 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:53.584508    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podStartSLOduration=2.0755290889999998 podStartE2EDuration="34.584491812s" totalImagesPullingTime="29.000389505s" totalInitContainerRuntime="3.508573218s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:20:53.534052514 +0000 UTC m=+3104.036206317" watchObservedRunningTime="2026-10-31 08:20:53.584491812 +0000 UTC m=+3104.086645598"
Oct 31 08:20:53.722703 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:53.722674    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:59.723064 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:59.723026    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:20:59.724981 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:20:59.724946    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:21:02.305437 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:02.305405    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"]
Oct 31 08:21:02.305855 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:02.305609    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" containerID="cri-o://4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872" gracePeriod=30
Oct 31 08:21:02.356278 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:02.356234    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"]
Oct 31 08:21:02.356800 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:02.356744    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" containerID="cri-o://77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec" gracePeriod=30
Oct 31 08:21:02.358025 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:02.357996    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"]
Oct 31 08:21:02.358430 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:02.358395    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" containerID="cri-o://37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1" gracePeriod=30
Oct 31 08:21:05.842736 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:05.842692    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:21:05.843110 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:05.842765    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:21:05.909323 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:05.909286    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-85kml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:21:05.909451 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:05.909359    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:21:05.921036 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:05.921006    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:21:05.921150 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:05.921059    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:21:08.102972 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.102943    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:21:08.145159 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.145134    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:21:08.185154 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.185127    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:21:08.217171 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217149    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-credential-socket\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.217256 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217186    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-certs\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.217256 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217211    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-credential-socket\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.217371 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217273    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-data\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.217371 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217292    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-token\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-token\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.217371 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217321    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-podinfo\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.217371 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217341    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-certs\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217380    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-socket\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217409    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-token\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-token\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217429    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-credential-socket" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217438    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c694012-1581-4ea4-aee0-35a6a108e0e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c694012-1581-4ea4-aee0-35a6a108e0e9-istiod-ca-cert\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217535    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-podinfo\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217546    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-certs" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217554    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-data" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.217619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217575    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-credential-socket" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217627    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-data\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217655    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-kube-api-access-95xpv\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-kube-api-access-95xpv\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217695    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-envoy\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217743    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-envoy\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217777    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/765fe623-41f8-4f3c-b44e-72293b72fa75-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765fe623-41f8-4f3c-b44e-72293b72fa75-istiod-ca-cert\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217785    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-certs" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217811    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-kube-api-access-rbxzq\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-kube-api-access-rbxzq\") pod \"765fe623-41f8-4f3c-b44e-72293b72fa75\" (UID: \"765fe623-41f8-4f3c-b44e-72293b72fa75\") "
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217815    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-socket" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217834    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c694012-1581-4ea4-aee0-35a6a108e0e9-istiod-ca-cert" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:21:08.218024 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.217848    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-socket\") pod \"2c694012-1581-4ea4-aee0-35a6a108e0e9\" (UID: \"2c694012-1581-4ea4-aee0-35a6a108e0e9\") "
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218145    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218164    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218177    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218190    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218203    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218218    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218231    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c694012-1581-4ea4-aee0-35a6a108e0e9-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218455    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-socket" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218551    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-data" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.218911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.218767    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/765fe623-41f8-4f3c-b44e-72293b72fa75-istiod-ca-cert" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:21:08.220290 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.220257    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-podinfo" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:21:08.220395 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.220359    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-token" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:21:08.220575 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.220538    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-kube-api-access-95xpv" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "kube-api-access-95xpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:21:08.220868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.220839    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-kube-api-access-rbxzq" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "kube-api-access-rbxzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:21:08.221071 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.221037    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-envoy" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.221177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.221119    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-podinfo" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:21:08.221292 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.221274    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-envoy" pod "765fe623-41f8-4f3c-b44e-72293b72fa75" (UID: "765fe623-41f8-4f3c-b44e-72293b72fa75"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.221592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.221576    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-token" pod "2c694012-1581-4ea4-aee0-35a6a108e0e9" (UID: "2c694012-1581-4ea4-aee0-35a6a108e0e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:21:08.319041 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.318966    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-socket\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319041 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319014    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-certs\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319061    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-token\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-token\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319082    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-envoy\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319126    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-data\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319157    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-podinfo\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319189    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istiod-ca-cert\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319222    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-credential-socket\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319259    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-kube-api-access-4kvxr\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-kube-api-access-4kvxr\") pod \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\" (UID: \"c9377b9c-f8a1-454a-bda9-69a6b53eef97\") "
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319292    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-socket" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319420    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-data" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319576    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319597    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319612    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/765fe623-41f8-4f3c-b44e-72293b72fa75-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319626    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbxzq\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-kube-api-access-rbxzq\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319641    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319655    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319652    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istiod-ca-cert" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319670    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.319689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319686    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.320251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319701    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.320251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319716    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.320251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319734    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c694012-1581-4ea4-aee0-35a6a108e0e9-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.320251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319747    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95xpv\" (UniqueName: \"kubernetes.io/projected/2c694012-1581-4ea4-aee0-35a6a108e0e9-kube-api-access-95xpv\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.320251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319761    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/765fe623-41f8-4f3c-b44e-72293b72fa75-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.320251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319869    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-credential-socket" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.320251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.319922    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-certs" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.321577 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.321551    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-envoy" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:21:08.321682 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.321648    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-kube-api-access-4kvxr" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "kube-api-access-4kvxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:21:08.321821 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.321805    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-podinfo" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:21:08.321875 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.321823    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-token" pod "c9377b9c-f8a1-454a-bda9-69a6b53eef97" (UID: "c9377b9c-f8a1-454a-bda9-69a6b53eef97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:21:08.420813 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.420784    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.420951 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.420816    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.420951 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.420832    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.420951 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.420846    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.420951 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.420858    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9377b9c-f8a1-454a-bda9-69a6b53eef97-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.420951 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.420870    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9377b9c-f8a1-454a-bda9-69a6b53eef97-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.420951 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.420879    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kvxr\" (UniqueName: \"kubernetes.io/projected/c9377b9c-f8a1-454a-bda9-69a6b53eef97-kube-api-access-4kvxr\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:21:08.443216 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.443191    4532 generic.go:356] "Generic (PLEG): container finished" podID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerID="37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1" exitCode=0
Oct 31 08:21:08.443358 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.443240    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"
Oct 31 08:21:08.443358 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.443268    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" event={"ID":"765fe623-41f8-4f3c-b44e-72293b72fa75","Type":"ContainerDied","Data":"37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1"}
Oct 31 08:21:08.443358 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.443320    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml" event={"ID":"765fe623-41f8-4f3c-b44e-72293b72fa75","Type":"ContainerDied","Data":"59ea488abfa4250415df26a6273aad33bf8f50f3ead0f786e4399f54f7312092"}
Oct 31 08:21:08.443358 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.443351    4532 scope.go:122] "RemoveContainer" containerID="37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1"
Oct 31 08:21:08.444686 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.444665    4532 generic.go:356] "Generic (PLEG): container finished" podID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerID="77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec" exitCode=0
Oct 31 08:21:08.444804 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.444714    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"
Oct 31 08:21:08.444804 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.444742    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" event={"ID":"c9377b9c-f8a1-454a-bda9-69a6b53eef97","Type":"ContainerDied","Data":"77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec"}
Oct 31 08:21:08.444804 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.444780    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5" event={"ID":"c9377b9c-f8a1-454a-bda9-69a6b53eef97","Type":"ContainerDied","Data":"e57b613ae4a906ad2f6930332bbc3a4d18903176a27d412445110174470fbebb"}
Oct 31 08:21:08.446541 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.446518    4532 generic.go:356] "Generic (PLEG): container finished" podID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerID="4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872" exitCode=0
Oct 31 08:21:08.446657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.446551    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" event={"ID":"2c694012-1581-4ea4-aee0-35a6a108e0e9","Type":"ContainerDied","Data":"4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872"}
Oct 31 08:21:08.446657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.446575    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"
Oct 31 08:21:08.446657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.446586    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk" event={"ID":"2c694012-1581-4ea4-aee0-35a6a108e0e9","Type":"ContainerDied","Data":"5aa4430c229d176eb69cc5e24f95d5ade135cd433858ce273d1abbd419080cef"}
Oct 31 08:21:08.456140 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.456120    4532 scope.go:122] "RemoveContainer" containerID="37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1"
Oct 31 08:21:08.456411 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:21:08.456390    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1\": container with ID starting with 37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1 not found: ID does not exist" containerID="37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1"
Oct 31 08:21:08.456510 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.456422    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1"} err="failed to get container status \"37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1\": rpc error: code = NotFound desc = could not find container \"37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1\": container with ID starting with 37edd13383d5b1cc433658fd16b5f3279449c18cf3bf13dff3efb5c3ea5b3da1 not found: ID does not exist"
Oct 31 08:21:08.456510 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.456443    4532 scope.go:122] "RemoveContainer" containerID="77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec"
Oct 31 08:21:08.465069 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.465053    4532 scope.go:122] "RemoveContainer" containerID="77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec"
Oct 31 08:21:08.465331 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:21:08.465313    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec\": container with ID starting with 77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec not found: ID does not exist" containerID="77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec"
Oct 31 08:21:08.465408 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.465341    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec"} err="failed to get container status \"77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec\": rpc error: code = NotFound desc = could not find container \"77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec\": container with ID starting with 77e494dc125e3a5afde6140d3f2ffc129abd5762ba84023d8929f177cc35b0ec not found: ID does not exist"
Oct 31 08:21:08.465408 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.465363    4532 scope.go:122] "RemoveContainer" containerID="4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872"
Oct 31 08:21:08.472310 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.472291    4532 scope.go:122] "RemoveContainer" containerID="4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872"
Oct 31 08:21:08.472573 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:21:08.472554    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872\": container with ID starting with 4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872 not found: ID does not exist" containerID="4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872"
Oct 31 08:21:08.472650 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.472579    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872"} err="failed to get container status \"4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872\": rpc error: code = NotFound desc = could not find container \"4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872\": container with ID starting with 4a3feb56a805e917af90f261a929205d1495024307f32e35570ea7b7bd9a7872 not found: ID does not exist"
Oct 31 08:21:08.608928 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.608822    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"]
Oct 31 08:21:08.612326 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.612299    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-85kml"]
Oct 31 08:21:08.720789 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.720757    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"]
Oct 31 08:21:08.726564 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.726537    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-4pkfk"]
Oct 31 08:21:08.834684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.834653    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"]
Oct 31 08:21:08.837956 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:08.837933    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-c4hh5"]
Oct 31 08:21:09.310899 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:21:09.310791    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:21:09.884301 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:09.624852    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" path="/var/lib/kubelet/pods/2c694012-1581-4ea4-aee0-35a6a108e0e9/volumes"
Oct 31 08:21:09.884301 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:09.625237    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" path="/var/lib/kubelet/pods/765fe623-41f8-4f3c-b44e-72293b72fa75/volumes"
Oct 31 08:21:09.884301 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:09.625626    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" path="/var/lib/kubelet/pods/c9377b9c-f8a1-454a-bda9-69a6b53eef97/volumes"
Oct 31 08:21:09.884301 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:09.723044    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:21:19.722919 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:19.722871    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:21:29.723152 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:29.723114    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:21:39.310050 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:21:39.310012    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:21:39.723307 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:39.723214    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:21:40.501346 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501310    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"]
Oct 31 08:21:40.501809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501742    4532 cpu_manager.go:414] "containerMap: removing container" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy"
Oct 31 08:21:40.501809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501754    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy"
Oct 31 08:21:40.501809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501781    4532 cpu_manager.go:414] "containerMap: removing container" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy"
Oct 31 08:21:40.501809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501786    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy"
Oct 31 08:21:40.501809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501800    4532 cpu_manager.go:414] "containerMap: removing container" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy"
Oct 31 08:21:40.501809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501806    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy"
Oct 31 08:21:40.501998 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501853    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9377b9c-f8a1-454a-bda9-69a6b53eef97" containerName="istio-proxy"
Oct 31 08:21:40.501998 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501860    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="765fe623-41f8-4f3c-b44e-72293b72fa75" containerName="istio-proxy"
Oct 31 08:21:40.501998 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.501871    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c694012-1581-4ea4-aee0-35a6a108e0e9" containerName="istio-proxy"
Oct 31 08:21:40.507404 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.507386    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.559681 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.559645    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Oct 31 08:21:40.583912 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.583877    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Oct 31 08:21:40.590608 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.590588    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-gw-ca-root-cert\""
Oct 31 08:21:40.590608 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.590593    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-gateway-1-openshift-default-dockercfg-8rb2k\""
Oct 31 08:21:40.610165 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.610135    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"]
Oct 31 08:21:40.685692 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685655    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-data\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.685820 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685708    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-certs\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.685820 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685788    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97a3aa47-da16-4cc0-b264-9a934856ced6-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.685896 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685822    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-token\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.685896 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685849    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-podinfo\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.685963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685929    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khfkg\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-kube-api-access-khfkg\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.685963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685949    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-socket\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.686028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685977    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-credential-socket\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.686028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.685995    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-envoy\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786386 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786353    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khfkg\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-kube-api-access-khfkg\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786395    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-socket\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786420    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-credential-socket\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786445    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-envoy\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786494    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-data\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786527    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-certs\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786553    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97a3aa47-da16-4cc0-b264-9a934856ced6-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786584    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-token\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786607    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-podinfo\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.786871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786826    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-socket\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.787033 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.786881    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-credential-socket\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.787033 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.787003    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-data\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.787147 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.787104    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-certs\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.787612 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.787588    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97a3aa47-da16-4cc0-b264-9a934856ced6-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.788859 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.788840    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-envoy\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.789011 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.788984    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-podinfo\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.842698 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.842669    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-token\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:40.842831 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:40.842779    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khfkg\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-kube-api-access-khfkg\") pod \"router-gateway-1-openshift-default-bf8d8c5df-xz2h9\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:41.118417 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:41.118333    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:41.536929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:41.536892    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"]
Oct 31 08:21:41.577614 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:21:41.577586    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a3aa47_da16_4cc0_b264_9a934856ced6.slice/crio-40b2407b8cf9c75a7dcac1ac9a9118c536b1ede24356c906b229c3a556de9434 WatchSource:0}: Error finding container 40b2407b8cf9c75a7dcac1ac9a9118c536b1ede24356c906b229c3a556de9434: Status 404 returned error can't find the container with id 40b2407b8cf9c75a7dcac1ac9a9118c536b1ede24356c906b229c3a556de9434
Oct 31 08:21:41.579840 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:41.579799    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:21:41.579936 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:41.579891    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:21:41.579980 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:41.579934    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:21:42.578892 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:42.578856    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" event={"ID":"97a3aa47-da16-4cc0-b264-9a934856ced6","Type":"ContainerStarted","Data":"4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb"}
Oct 31 08:21:42.579368 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:42.578918    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" event={"ID":"97a3aa47-da16-4cc0-b264-9a934856ced6","Type":"ContainerStarted","Data":"40b2407b8cf9c75a7dcac1ac9a9118c536b1ede24356c906b229c3a556de9434"}
Oct 31 08:21:42.744436 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:42.744386    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" podStartSLOduration=2.744373813 podStartE2EDuration="2.744373813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:21:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:21:42.743211902 +0000 UTC m=+3153.245365693" watchObservedRunningTime="2026-10-31 08:21:42.744373813 +0000 UTC m=+3153.246527603"
Oct 31 08:21:43.119149 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:43.119115    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:43.124051 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:43.124028    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:43.583957 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:43.583920    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:43.584960 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:43.584935    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:49.723044 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:49.722993    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:21:53.633429 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:53.633399    4532 generic.go:356] "Generic (PLEG): container finished" podID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerID="4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb" exitCode=0
Oct 31 08:21:53.633877 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:53.633477    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" event={"ID":"97a3aa47-da16-4cc0-b264-9a934856ced6","Type":"ContainerDied","Data":"4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb"}
Oct 31 08:21:53.633877 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:53.633769    4532 scope.go:122] "RemoveContainer" containerID="4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb"
Oct 31 08:21:53.634871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:53.634844    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:21:53.634933 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:53.634917    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:21:53.634969 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:53.634952    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:21:54.640037 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:54.639999    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" event={"ID":"97a3aa47-da16-4cc0-b264-9a934856ced6","Type":"ContainerStarted","Data":"1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6"}
Oct 31 08:21:55.118983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:55.118941    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:55.120043 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:55.120012    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 31 08:21:56.119126 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:56.119074    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 31 08:21:57.118958 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:57.118916    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 31 08:21:58.119438 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:58.119392    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 31 08:21:58.585305 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:58.584557    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:21:59.119030 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:59.118976    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 31 08:21:59.723039 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:21:59.722998    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:22:00.123236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:00.123198    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:22:00.124040 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:00.124016    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:22:09.309906 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:22:09.309870    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:22:09.732258 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:09.732195    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:22:09.739586 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:09.739557    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:22:36.553396 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:36.553305    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"]
Oct 31 08:22:39.310813 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:22:39.310781    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:22:51.576190 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:51.576128    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" containerID="cri-o://2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6" gracePeriod=45
Oct 31 08:22:51.733570 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:51.733525    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": dial tcp 10.133.0.45:8000: connect: connection refused"
Oct 31 08:22:52.220223 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.220198    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:22:52.300948 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.300920    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-model-cache\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301067 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.300985    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-kserve-provision-location\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301067 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301013    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-home\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-home\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301067 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301054    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-dshm\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301240 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301134    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96704e56-2c3f-451e-8595-208e419b07b9-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96704e56-2c3f-451e-8595-208e419b07b9-cabundle-cert\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301240 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301173    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-tmp-dir\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301240 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301199    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-model-cache" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:22:52.301392 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301254    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96704e56-2c3f-451e-8595-208e419b07b9-tls-certs\" (UniqueName: \"kubernetes.io/secret/96704e56-2c3f-451e-8595-208e419b07b9-tls-certs\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301392 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301282    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96704e56-2c3f-451e-8595-208e419b07b9-kube-api-access-dm4df\" (UniqueName: \"kubernetes.io/projected/96704e56-2c3f-451e-8595-208e419b07b9-kube-api-access-dm4df\") pod \"96704e56-2c3f-451e-8595-208e419b07b9\" (UID: \"96704e56-2c3f-451e-8595-208e419b07b9\") "
Oct 31 08:22:52.301533 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301487    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-home" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:22:52.301574 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301527    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96704e56-2c3f-451e-8595-208e419b07b9-cabundle-cert" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:22:52.301574 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.301550    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.303609 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.303584    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96704e56-2c3f-451e-8595-208e419b07b9-tls-certs" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:22:52.303609 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.303592    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-dshm" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:22:52.303962 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.303944    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96704e56-2c3f-451e-8595-208e419b07b9-kube-api-access-dm4df" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "kube-api-access-dm4df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:22:52.313092 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.313069    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-tmp-dir" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:22:52.321330 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.321310    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-kserve-provision-location" pod "96704e56-2c3f-451e-8595-208e419b07b9" (UID: "96704e56-2c3f-451e-8595-208e419b07b9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:22:52.402106 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.402084    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.402106 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.402104    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.402224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.402114    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.402224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.402123    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96704e56-2c3f-451e-8595-208e419b07b9-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.402224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.402132    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96704e56-2c3f-451e-8595-208e419b07b9-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.402224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.402140    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96704e56-2c3f-451e-8595-208e419b07b9-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.402224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.402147    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm4df\" (UniqueName: \"kubernetes.io/projected/96704e56-2c3f-451e-8595-208e419b07b9-kube-api-access-dm4df\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:22:52.876800 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.876768    4532 generic.go:356] "Generic (PLEG): container finished" podID="96704e56-2c3f-451e-8595-208e419b07b9" containerID="2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6" exitCode=0
Oct 31 08:22:52.877241 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.876830    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"
Oct 31 08:22:52.877241 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.876837    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" event={"ID":"96704e56-2c3f-451e-8595-208e419b07b9","Type":"ContainerDied","Data":"2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6"}
Oct 31 08:22:52.877241 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.876900    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt" event={"ID":"96704e56-2c3f-451e-8595-208e419b07b9","Type":"ContainerDied","Data":"e5681c277f2b8100c91deb2bf49de1460a3f2ee5aeff315187ae599a457c5161"}
Oct 31 08:22:52.877241 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.876921    4532 scope.go:122] "RemoveContainer" containerID="2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6"
Oct 31 08:22:52.885313 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.885296    4532 scope.go:122] "RemoveContainer" containerID="db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b"
Oct 31 08:22:52.892833 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.892813    4532 scope.go:122] "RemoveContainer" containerID="2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6"
Oct 31 08:22:52.893096 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:22:52.893077    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6\": container with ID starting with 2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6 not found: ID does not exist" containerID="2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6"
Oct 31 08:22:52.893134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.893104    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6"} err="failed to get container status \"2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6\": rpc error: code = NotFound desc = could not find container \"2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6\": container with ID starting with 2906c9a9449c4ac0e8684931e41d381b6f3d8c53b4b2e21eceec49f1be876ba6 not found: ID does not exist"
Oct 31 08:22:52.893134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.893122    4532 scope.go:122] "RemoveContainer" containerID="db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b"
Oct 31 08:22:52.893367 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:22:52.893346    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b\": container with ID starting with db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b not found: ID does not exist" containerID="db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b"
Oct 31 08:22:52.893425 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:52.893377    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b"} err="failed to get container status \"db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b\": rpc error: code = NotFound desc = could not find container \"db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b\": container with ID starting with db2c613a4f8888e4aaef463d1107ed068d71a349b7a0ff4a8cddefc9c4e5ce5b not found: ID does not exist"
Oct 31 08:22:53.038124 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:53.038097    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"]
Oct 31 08:22:53.088096 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:53.088072    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-685b58654f-587kt"]
Oct 31 08:22:53.623433 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:22:53.623395    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96704e56-2c3f-451e-8595-208e419b07b9" path="/var/lib/kubelet/pods/96704e56-2c3f-451e-8595-208e419b07b9/volumes"
Oct 31 08:23:09.310671 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:23:09.310637    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:23:31.149386 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.149352    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"]
Oct 31 08:23:31.150007 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.149808    4532 cpu_manager.go:414] "containerMap: removing container" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="storage-initializer"
Oct 31 08:23:31.150007 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.149828    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="storage-initializer"
Oct 31 08:23:31.150007 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.149870    4532 cpu_manager.go:414] "containerMap: removing container" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main"
Oct 31 08:23:31.150007 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.149878    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main"
Oct 31 08:23:31.150007 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.149954    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="96704e56-2c3f-451e-8595-208e419b07b9" containerName="main"
Oct 31 08:23:31.152781 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.152760    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.193043 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.193006    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"]
Oct 31 08:23:31.240929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.240895    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"default-dockercfg-hgk7x\""
Oct 31 08:23:31.240929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.240895    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"seaweedfs-s3-creds\""
Oct 31 08:23:31.240929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.240895    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"openshift-service-ca.crt\""
Oct 31 08:23:31.241121 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.240895    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:23:31.241121 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.240983    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Oct 31 08:23:31.241121 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.241067    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"kube-root-ca.crt\""
Oct 31 08:23:31.291758 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.291729    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7f72bf55-ee0f-467e-8b19-678ce17f17a5-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.291887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.291761    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4d2z\" (UniqueName: \"kubernetes.io/projected/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kube-api-access-w4d2z\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.291887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.291786    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.291887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.291849    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.291887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.291881    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.292023 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.291969    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-home\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.292023 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.292010    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-dshm\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.292088 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.292025    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393341 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393310    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-dshm\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393341 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393341    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393369    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7f72bf55-ee0f-467e-8b19-678ce17f17a5-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393400    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4d2z\" (UniqueName: \"kubernetes.io/projected/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kube-api-access-w4d2z\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393433    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393460    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393508    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393544    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-home\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393969 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393938    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-model-cache\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.393969 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.393958    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.394150 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.394003    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-home\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.394150 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.394063    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.394250 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.394227    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7f72bf55-ee0f-467e-8b19-678ce17f17a5-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.395795 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.395770    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-dshm\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.395973 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.395955    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.449112 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.449060    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4d2z\" (UniqueName: \"kubernetes.io/projected/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kube-api-access-w4d2z\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.467918 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.467897    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:31.888544 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:31.888511    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"]
Oct 31 08:23:31.891318 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:23:31.891293    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f72bf55_ee0f_467e_8b19_678ce17f17a5.slice/crio-6f94fdd6b8174553cbf61570f197b72a7a1242b630e0199d5b1de46f9a6b68c4 WatchSource:0}: Error finding container 6f94fdd6b8174553cbf61570f197b72a7a1242b630e0199d5b1de46f9a6b68c4: Status 404 returned error can't find the container with id 6f94fdd6b8174553cbf61570f197b72a7a1242b630e0199d5b1de46f9a6b68c4
Oct 31 08:23:32.024458 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:32.024419    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" event={"ID":"7f72bf55-ee0f-467e-8b19-678ce17f17a5","Type":"ContainerStarted","Data":"09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65"}
Oct 31 08:23:32.024637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:32.024500    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" event={"ID":"7f72bf55-ee0f-467e-8b19-678ce17f17a5","Type":"ContainerStarted","Data":"6f94fdd6b8174553cbf61570f197b72a7a1242b630e0199d5b1de46f9a6b68c4"}
Oct 31 08:23:36.042288 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:36.042257    4532 generic.go:356] "Generic (PLEG): container finished" podID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerID="09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65" exitCode=0
Oct 31 08:23:36.042803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:36.042337    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" event={"ID":"7f72bf55-ee0f-467e-8b19-678ce17f17a5","Type":"ContainerDied","Data":"09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65"}
Oct 31 08:23:37.048295 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:37.048260    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" event={"ID":"7f72bf55-ee0f-467e-8b19-678ce17f17a5","Type":"ContainerStarted","Data":"d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43"}
Oct 31 08:23:37.262251 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:37.262201    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podStartSLOduration=2.790126448 podStartE2EDuration="6.262188616s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.472062168s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:23:37.262062517 +0000 UTC m=+3267.764216306" watchObservedRunningTime="2026-10-31 08:23:37.262188616 +0000 UTC m=+3267.764342413"
Oct 31 08:23:37.468843 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:37.468768    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:39.311286 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:23:39.311247    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:23:41.469031 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:41.468995    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:23:41.470572 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:41.470541    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": dial tcp 10.133.0.47:8000: connect: connection refused"
Oct 31 08:23:48.563974 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.563939    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"]
Oct 31 08:23:48.565892 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.565875    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.613917 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.613888    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"]
Oct 31 08:23:48.615794 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.615772    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"]
Oct 31 08:23:48.615920 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.615904    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.617682 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.617660    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.624803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.624781    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 08:23:48.661264 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.661232    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 08:23:48.661410 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.661279    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 08:23:48.661502 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.661417    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-z29g8\""
Oct 31 08:23:48.663019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.662992    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"]
Oct 31 08:23:48.676608 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.676585    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"]
Oct 31 08:23:48.719919 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.719891    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"]
Oct 31 08:23:48.739593 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739559    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.739593 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739594    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxsd5\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-kube-api-access-dxsd5\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.739798 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739632    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.739798 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739706    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.739798 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739781    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.739911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739857    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.739911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739878    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.739911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739895    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.740034 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739935    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5b7j\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-kube-api-access-v5b7j\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.740034 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.739966    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.740138 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740067    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.740138 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740107    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.740217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740160    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.740217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740181    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.740217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740207    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.740332 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740293    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.740369 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740354    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqz5x\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-kube-api-access-kqz5x\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.740402 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740380    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.740435 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740399    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.740435 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740425    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.740532 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740490    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.740532 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740528    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.740605 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740579    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.740605 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740599    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.740681 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740620    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.740681 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740637    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.740746 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.740697    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.841493 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841401    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.841493 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841451    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.841718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841504    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.841718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841561    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.841718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841589    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.841718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841616    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.841718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841642    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5b7j\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-kube-api-access-v5b7j\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.841718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841668    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.841718 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841699    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841730    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841759    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841791    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841831    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841862    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841895    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqz5x\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-kube-api-access-kqz5x\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841934    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841965    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.841996    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842021    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842048    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.842078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842076    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842079    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842346    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842385    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842407    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842083    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842412    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842500    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842534    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842562    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxsd5\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-kube-api-access-dxsd5\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.842719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842678    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.843236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842930    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.843236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.842997    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.843236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843118    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.843236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843164    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.843236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843190    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.843514 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843266    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.843514 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843420    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.843636 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843518    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.843711 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843688    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.843769 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843712    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.843769 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.843750    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.844385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.844350    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.844385 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.844383    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.845964 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.845935    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.846264 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.846245    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.846348 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.846295    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.846348 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.846305    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.899911 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.899882    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.900443 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.900416    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.900568 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.900453    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxsd5\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-kube-api-access-dxsd5\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:48.900568 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.900419    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.900568 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.900555    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5b7j\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-kube-api-access-v5b7j\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.900691 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.900669    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqz5x\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-kube-api-access-kqz5x\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-26jsf\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:48.928674 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.928650    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:48.934921 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:48.934901    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:49.177520 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.177435    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:49.357004 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.356978    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"]
Oct 31 08:23:49.357903 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:23:49.357878    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8061b9_ac7a_49eb_8ade_958158f44d7c.slice/crio-779bdfb5606277e9e2c708e602ac2792d021228b4156337bfd7b045ec4a75be3 WatchSource:0}: Error finding container 779bdfb5606277e9e2c708e602ac2792d021228b4156337bfd7b045ec4a75be3: Status 404 returned error can't find the container with id 779bdfb5606277e9e2c708e602ac2792d021228b4156337bfd7b045ec4a75be3
Oct 31 08:23:49.358697 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:23:49.358676    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9051cc9_6e30_4b4d_bf3f_a792713499c5.slice/crio-2569c73e0c4245c988e588f2f58a09ad11dae67a58885e5f94f7851de4e190bd WatchSource:0}: Error finding container 2569c73e0c4245c988e588f2f58a09ad11dae67a58885e5f94f7851de4e190bd: Status 404 returned error can't find the container with id 2569c73e0c4245c988e588f2f58a09ad11dae67a58885e5f94f7851de4e190bd
Oct 31 08:23:49.358927 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.358890    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"]
Oct 31 08:23:49.360594 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.360553    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.360663 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.360628    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.360716 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.360665    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.360918 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.360886    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.360984 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.360962    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.361083 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.361015    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.550534 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.550477    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"]
Oct 31 08:23:49.553354 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:23:49.553318    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2bce18f_6dd2_44f8_81b4_166bff8123a1.slice/crio-2af3c597289ead1dbe38993aab5a01d8e18ae1cf784982e83eff7a6a66c1ac1d WatchSource:0}: Error finding container 2af3c597289ead1dbe38993aab5a01d8e18ae1cf784982e83eff7a6a66c1ac1d: Status 404 returned error can't find the container with id 2af3c597289ead1dbe38993aab5a01d8e18ae1cf784982e83eff7a6a66c1ac1d
Oct 31 08:23:49.555816 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.555779    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.555910 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.555850    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:49.555910 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:49.555881    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:23:50.115487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.114897    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" event={"ID":"d9051cc9-6e30-4b4d-bf3f-a792713499c5","Type":"ContainerStarted","Data":"a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316"}
Oct 31 08:23:50.115487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.114968    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" event={"ID":"d9051cc9-6e30-4b4d-bf3f-a792713499c5","Type":"ContainerStarted","Data":"2569c73e0c4245c988e588f2f58a09ad11dae67a58885e5f94f7851de4e190bd"}
Oct 31 08:23:50.128489 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.127251    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" event={"ID":"1b8061b9-ac7a-49eb-8ade-958158f44d7c","Type":"ContainerStarted","Data":"2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9"}
Oct 31 08:23:50.128489 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.127322    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" event={"ID":"1b8061b9-ac7a-49eb-8ade-958158f44d7c","Type":"ContainerStarted","Data":"779bdfb5606277e9e2c708e602ac2792d021228b4156337bfd7b045ec4a75be3"}
Oct 31 08:23:50.130487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.129730    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" event={"ID":"b2bce18f-6dd2-44f8-81b4-166bff8123a1","Type":"ContainerStarted","Data":"5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82"}
Oct 31 08:23:50.130487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.129786    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" event={"ID":"b2bce18f-6dd2-44f8-81b4-166bff8123a1","Type":"ContainerStarted","Data":"2af3c597289ead1dbe38993aab5a01d8e18ae1cf784982e83eff7a6a66c1ac1d"}
Oct 31 08:23:50.183487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.182916    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:50.183487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.183059    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 31 08:23:50.183487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.183132    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 31 08:23:50.249530 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.248591    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podStartSLOduration=2.248572178 podStartE2EDuration="2.248572178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:23:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:23:50.247010893 +0000 UTC m=+3280.749164683" watchObservedRunningTime="2026-10-31 08:23:50.248572178 +0000 UTC m=+3280.750725968"
Oct 31 08:23:50.385559 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.380700    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podStartSLOduration=2.380680361 podStartE2EDuration="2.380680361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:23:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:23:50.378983223 +0000 UTC m=+3280.881137020" watchObservedRunningTime="2026-10-31 08:23:50.380680361 +0000 UTC m=+3280.882834151"
Oct 31 08:23:50.931279 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.931242    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:50.931680 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.931655    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 08:23:50.931846 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.931821    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 08:23:50.936796 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.936771    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:50.938672 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.938565    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 08:23:50.938672 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:50.938612    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 08:23:51.179922 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:51.179885    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 31 08:23:51.180542 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:51.180512    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 31 08:23:51.469068 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:51.469022    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": dial tcp 10.133.0.47:8000: connect: connection refused"
Oct 31 08:23:51.933932 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:51.933882    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 08:23:51.934140 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:51.933953    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 08:23:51.937497 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:51.935822    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 08:23:51.937497 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:51.935869    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 08:23:52.178486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:52.177874    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 31 08:23:52.178486 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:52.177932    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 31 08:23:52.931491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:52.930030    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 08:23:52.931491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:52.930110    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 08:23:52.936891 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:52.936792    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 08:23:52.936891 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:52.936838    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 08:23:53.178946 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:53.178910    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 31 08:23:53.179262 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:53.179213    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 31 08:23:53.931579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:53.931529    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 08:23:53.932204 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:53.932152    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 08:23:53.935756 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:53.935735    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 08:23:53.935961 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:53.935940    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 08:23:54.179240 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:54.178812    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 31 08:23:54.179240 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:54.178867    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 31 08:23:54.931489 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:54.930847    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 08:23:54.931489 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:54.930908    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 08:23:54.937581 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:54.937551    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 08:23:54.937801 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:54.937774    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 08:23:55.180976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.180940    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 31 08:23:55.181270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.181231    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 31 08:23:55.932830 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.932798    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:55.933314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.933050    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:55.933732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.933716    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:23:55.939583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.939563    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:55.939824 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.939803    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:55.940350 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:55.940333    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:23:56.097608 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:56.097551    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podStartSLOduration=8.097536754 podStartE2EDuration="8.097536754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:23:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:23:50.509489102 +0000 UTC m=+3281.011642896" watchObservedRunningTime="2026-10-31 08:23:56.097536754 +0000 UTC m=+3286.599690544"
Oct 31 08:23:56.182010 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:56.181981    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:56.182294 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:56.182272    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:23:56.183039 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:23:56.182983    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:24:01.469198 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:01.469156    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": dial tcp 10.133.0.47:8000: connect: connection refused"
Oct 31 08:24:09.312727 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:24:09.312683    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:24:11.468902 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:11.468858    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": dial tcp 10.133.0.47:8000: connect: connection refused"
Oct 31 08:24:21.468904 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:21.468862    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": dial tcp 10.133.0.47:8000: connect: connection refused"
Oct 31 08:24:31.468768 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:31.468724    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": dial tcp 10.133.0.47:8000: connect: connection refused"
Oct 31 08:24:39.310564 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:24:39.310528    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:24:41.468634 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:41.468599    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.47:8000/health\": dial tcp 10.133.0.47:8000: connect: connection refused"
Oct 31 08:24:50.989379 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:50.989343    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"]
Oct 31 08:24:50.989919 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:50.989596    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy" containerID="cri-o://1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6" gracePeriod=30
Oct 31 08:24:51.478546 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:51.478518    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:24:51.486255 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:51.486229    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:24:56.345762 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.345735    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:24:56.444871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.444839    4532 generic.go:356] "Generic (PLEG): container finished" podID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerID="1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6" exitCode=0
Oct 31 08:24:56.445021 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.444886    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"
Oct 31 08:24:56.445021 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.444917    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" event={"ID":"97a3aa47-da16-4cc0-b264-9a934856ced6","Type":"ContainerDied","Data":"1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6"}
Oct 31 08:24:56.445021 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.444976    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9" event={"ID":"97a3aa47-da16-4cc0-b264-9a934856ced6","Type":"ContainerDied","Data":"40b2407b8cf9c75a7dcac1ac9a9118c536b1ede24356c906b229c3a556de9434"}
Oct 31 08:24:56.445021 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.445006    4532 scope.go:122] "RemoveContainer" containerID="1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6"
Oct 31 08:24:56.453088 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.453072    4532 scope.go:122] "RemoveContainer" containerID="4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb"
Oct 31 08:24:56.460175 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.460160    4532 scope.go:122] "RemoveContainer" containerID="1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6"
Oct 31 08:24:56.460401 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:24:56.460381    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6\": container with ID starting with 1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6 not found: ID does not exist" containerID="1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6"
Oct 31 08:24:56.460511 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.460415    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6"} err="failed to get container status \"1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6\": rpc error: code = NotFound desc = could not find container \"1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6\": container with ID starting with 1d6cc31baba5f7e9a243f24938d91ac1b9fb86d9006c8f37a80b02f6964678d6 not found: ID does not exist"
Oct 31 08:24:56.460511 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.460438    4532 scope.go:122] "RemoveContainer" containerID="4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb"
Oct 31 08:24:56.460712 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:24:56.460693    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb\": container with ID starting with 4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb not found: ID does not exist" containerID="4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb"
Oct 31 08:24:56.460772 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.460722    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb"} err="failed to get container status \"4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb\": rpc error: code = NotFound desc = could not find container \"4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb\": container with ID starting with 4455b59a1575f402586788e575c0ab394344aa4bf0b9e9e7c65eb42c297729bb not found: ID does not exist"
Oct 31 08:24:56.497455 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497432    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97a3aa47-da16-4cc0-b264-9a934856ced6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97a3aa47-da16-4cc0-b264-9a934856ced6-istiod-ca-cert\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497578 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497486    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-socket\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497578 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497535    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-kube-api-access-khfkg\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-kube-api-access-khfkg\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497578 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497569    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-podinfo\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497592    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-token\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-token\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497617    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-envoy\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497684    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-data\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497717    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-certs\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497750    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-credential-socket\") pod \"97a3aa47-da16-4cc0-b264-9a934856ced6\" (UID: \"97a3aa47-da16-4cc0-b264-9a934856ced6\") "
Oct 31 08:24:56.497952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.497838    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97a3aa47-da16-4cc0-b264-9a934856ced6-istiod-ca-cert" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:24:56.498128 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.498081    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97a3aa47-da16-4cc0-b264-9a934856ced6-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.498128 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.498116    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-data" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:24:56.498297 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.498162    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-certs" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:24:56.498380 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.498361    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-socket" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:24:56.498587 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.498563    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-credential-socket" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:24:56.500231 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.500199    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-envoy" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:24:56.500325 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.500296    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-podinfo" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:24:56.500395 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.500363    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-kube-api-access-khfkg" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "kube-api-access-khfkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:24:56.500677 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.500658    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-token" pod "97a3aa47-da16-4cc0-b264-9a934856ced6" (UID: "97a3aa47-da16-4cc0-b264-9a934856ced6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:24:56.599619 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599581    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.599782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599622    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khfkg\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-kube-api-access-khfkg\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.599782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599641    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.599782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599656    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.599782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599672    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.599782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599687    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.599782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599703    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.599782 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.599719    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97a3aa47-da16-4cc0-b264-9a934856ced6-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:24:56.865562 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.865484    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"]
Oct 31 08:24:56.873796 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:56.873766    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-bf8d8c5df-xz2h9"]
Oct 31 08:24:57.625072 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:57.625036    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" path="/var/lib/kubelet/pods/97a3aa47-da16-4cc0-b264-9a934856ced6/volumes"
Oct 31 08:24:58.602604 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:24:58.602572    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"]
Oct 31 08:25:09.310654 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:25:09.310618    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:25:13.624713 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:13.624673    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main" containerID="cri-o://d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43" gracePeriod=45
Oct 31 08:25:14.178667 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.178645    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:25:14.342697 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.342667    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kube-api-access-w4d2z\" (UniqueName: \"kubernetes.io/projected/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kube-api-access-w4d2z\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.342890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.342711    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-home\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-home\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.342890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.342743    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tls-certs\" (UniqueName: \"kubernetes.io/secret/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tls-certs\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.343094 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343030    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kserve-provision-location\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.343094 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343075    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tmp-dir\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.343211 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343119    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-dshm\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.343211 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343167    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-model-cache\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.343325 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343225    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f72bf55-ee0f-467e-8b19-678ce17f17a5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7f72bf55-ee0f-467e-8b19-678ce17f17a5-cabundle-cert\") pod \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\" (UID: \"7f72bf55-ee0f-467e-8b19-678ce17f17a5\") "
Oct 31 08:25:14.343325 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343161    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-home" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:25:14.343561 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343513    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-model-cache" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:25:14.343754 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343724    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tmp-dir" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:25:14.343851 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.343789    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f72bf55-ee0f-467e-8b19-678ce17f17a5-cabundle-cert" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:25:14.344133 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.344111    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.344210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.344138    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7f72bf55-ee0f-467e-8b19-678ce17f17a5-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.344210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.344152    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.344210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.344166    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.345250 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.345223    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tls-certs" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:25:14.345339 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.345306    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kube-api-access-w4d2z" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "kube-api-access-w4d2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:25:14.345339 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.345327    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-dshm" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:25:14.366400 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.366376    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kserve-provision-location" pod "7f72bf55-ee0f-467e-8b19-678ce17f17a5" (UID: "7f72bf55-ee0f-467e-8b19-678ce17f17a5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:25:14.445268 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.445241    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4d2z\" (UniqueName: \"kubernetes.io/projected/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kube-api-access-w4d2z\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.445268 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.445266    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f72bf55-ee0f-467e-8b19-678ce17f17a5-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.445446 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.445277    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.445446 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.445286    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f72bf55-ee0f-467e-8b19-678ce17f17a5-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:25:14.523048 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.523020    4532 generic.go:356] "Generic (PLEG): container finished" podID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerID="d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43" exitCode=0
Oct 31 08:25:14.523187 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.523062    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" event={"ID":"7f72bf55-ee0f-467e-8b19-678ce17f17a5","Type":"ContainerDied","Data":"d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43"}
Oct 31 08:25:14.523187 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.523093    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"
Oct 31 08:25:14.523187 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.523111    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq" event={"ID":"7f72bf55-ee0f-467e-8b19-678ce17f17a5","Type":"ContainerDied","Data":"6f94fdd6b8174553cbf61570f197b72a7a1242b630e0199d5b1de46f9a6b68c4"}
Oct 31 08:25:14.523187 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.523133    4532 scope.go:122] "RemoveContainer" containerID="d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43"
Oct 31 08:25:14.532088 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.532071    4532 scope.go:122] "RemoveContainer" containerID="09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65"
Oct 31 08:25:14.539875 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.539858    4532 scope.go:122] "RemoveContainer" containerID="d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43"
Oct 31 08:25:14.540148 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:25:14.540128    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43\": container with ID starting with d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43 not found: ID does not exist" containerID="d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43"
Oct 31 08:25:14.540190 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.540157    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43"} err="failed to get container status \"d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43\": rpc error: code = NotFound desc = could not find container \"d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43\": container with ID starting with d3200427c52f7dce59da9f9cb2b3226835e07abd8d6e4e335126344585c55c43 not found: ID does not exist"
Oct 31 08:25:14.540190 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.540175    4532 scope.go:122] "RemoveContainer" containerID="09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65"
Oct 31 08:25:14.540407 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:25:14.540387    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65\": container with ID starting with 09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65 not found: ID does not exist" containerID="09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65"
Oct 31 08:25:14.540484 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.540416    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65"} err="failed to get container status \"09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65\": rpc error: code = NotFound desc = could not find container \"09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65\": container with ID starting with 09c7e0552e6117557a85ac8c2777845d9eb00193e885123af3b888a165b3de65 not found: ID does not exist"
Oct 31 08:25:14.684928 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.684862    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"]
Oct 31 08:25:14.688971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:14.688947    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-555c887948-pp9cq"]
Oct 31 08:25:15.624643 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:15.624611    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" path="/var/lib/kubelet/pods/7f72bf55-ee0f-467e-8b19-678ce17f17a5/volumes"
Oct 31 08:25:39.309923 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:25:39.309888    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:25:51.709865 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.709827    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"]
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710160    4532 cpu_manager.go:414] "containerMap: removing container" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710172    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710181    4532 cpu_manager.go:414] "containerMap: removing container" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710186    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710195    4532 cpu_manager.go:414] "containerMap: removing container" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710200    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710213    4532 cpu_manager.go:414] "containerMap: removing container" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="storage-initializer"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710221    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="storage-initializer"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710288    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710296    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="97a3aa47-da16-4cc0-b264-9a934856ced6" containerName="istio-proxy"
Oct 31 08:25:51.710421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.710323    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f72bf55-ee0f-467e-8b19-678ce17f17a5" containerName="main"
Oct 31 08:25:51.713586 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.713565    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.767009 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.766977    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"]
Oct 31 08:25:51.808749 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808723    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kserve-provision-location\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.808881 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808760    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tmp-dir\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.808881 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808777    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tls-certs\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.808881 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808808    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7h8r\" (UniqueName: \"kubernetes.io/projected/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kube-api-access-g7h8r\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.808997 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808897    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-cabundle-cert\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.808997 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808924    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-model-cache\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.808997 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808938    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-home\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.808997 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.808968    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-dshm\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.814933 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.814910    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Oct 31 08:25:51.815047 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.814910    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"seaweedfs-s3-creds\""
Oct 31 08:25:51.815047 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.814988    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Oct 31 08:25:51.815177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.815154    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Oct 31 08:25:51.815177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.815162    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-fwsxf\""
Oct 31 08:25:51.815177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.815166    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:25:51.909556 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909527    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-model-cache\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.909556 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909559    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-home\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.909750 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909596    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-dshm\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.909750 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909633    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kserve-provision-location\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.909750 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909657    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tmp-dir\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.909750 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909676    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tls-certs\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.909750 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909697    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7h8r\" (UniqueName: \"kubernetes.io/projected/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kube-api-access-g7h8r\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.909750 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909736    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-cabundle-cert\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.910029 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.909999    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-home\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.910095 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.910068    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-model-cache\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.910184 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.910164    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tmp-dir\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.910263 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.910243    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kserve-provision-location\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.910496 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.910456    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-cabundle-cert\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.912533 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.912513    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-dshm\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.912942 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.912923    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tls-certs\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:51.965604 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:51.965550    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7h8r\" (UniqueName: \"kubernetes.io/projected/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kube-api-access-g7h8r\") pod \"stop-feature-test-kserve-6df697898d-d9kt6\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:52.029284 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:52.029262    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:25:52.500737 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:52.500712    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"]
Oct 31 08:25:52.502684 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:25:52.502658    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4cf078_2a0c_4e85_a11c_2f012f4f49cb.slice/crio-46d5eefd13d50e883a49543c03eabf30453aee8e9d6fdd2197df3b99ea0453b7 WatchSource:0}: Error finding container 46d5eefd13d50e883a49543c03eabf30453aee8e9d6fdd2197df3b99ea0453b7: Status 404 returned error can't find the container with id 46d5eefd13d50e883a49543c03eabf30453aee8e9d6fdd2197df3b99ea0453b7
Oct 31 08:25:52.669340 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:52.669301    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" event={"ID":"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb","Type":"ContainerStarted","Data":"a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838"}
Oct 31 08:25:52.669559 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:52.669363    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" event={"ID":"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb","Type":"ContainerStarted","Data":"46d5eefd13d50e883a49543c03eabf30453aee8e9d6fdd2197df3b99ea0453b7"}
Oct 31 08:25:55.684956 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:55.684923    4532 generic.go:356] "Generic (PLEG): container finished" podID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerID="a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838" exitCode=0
Oct 31 08:25:55.685426 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:55.684995    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" event={"ID":"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb","Type":"ContainerDied","Data":"a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838"}
Oct 31 08:25:56.692097 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:56.692064    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" event={"ID":"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb","Type":"ContainerStarted","Data":"8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9"}
Oct 31 08:25:56.908606 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:56.908557    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podStartSLOduration=3.709920703 podStartE2EDuration="5.908544082s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.198623379s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:25:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:25:56.858450983 +0000 UTC m=+3407.360604775" watchObservedRunningTime="2026-10-31 08:25:56.908544082 +0000 UTC m=+3407.410697869"
Oct 31 08:25:57.030127 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:25:57.030102    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:26:02.029420 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:26:02.029385    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:26:02.030917 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:26:02.030891    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:26:09.310818 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:26:09.310777    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:26:12.030563 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:26:12.030520    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:26:22.030458 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:26:22.030413    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:26:32.030399 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:26:32.030353    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:26:39.310355 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:26:39.310317    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:26:42.030631 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:26:42.030589    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:26:52.030271 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:26:52.030189    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:27:02.029680 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:02.029635    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:27:09.310817 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:27:09.310772    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:27:12.030437 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:12.030394    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:27:22.030070 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:22.030016    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:27:32.030017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:32.029978    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:27:39.320128 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:27:39.320091    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:27:42.039229 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:42.039200    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:27:42.046827 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:42.046800    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:27:48.786805 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:48.786769    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"]
Oct 31 08:27:56.519969 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.519937    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"]
Oct 31 08:27:56.522968 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.522948    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.625391 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.625355    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"]
Oct 31 08:27:56.667364 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667336    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-model-cache\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.667513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667379    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570ff1cb-b6dd-4045-859c-dc59335074f6-tls-certs\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.667513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667401    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-kserve-provision-location\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.667513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667493    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-dshm\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.667639 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667524    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msntq\" (UniqueName: \"kubernetes.io/projected/570ff1cb-b6dd-4045-859c-dc59335074f6-kube-api-access-msntq\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.667639 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667543    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-tmp-dir\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.667639 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667559    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/570ff1cb-b6dd-4045-859c-dc59335074f6-cabundle-cert\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.667753 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.667651    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-home\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.768734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768707    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-dshm\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.768734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768738    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msntq\" (UniqueName: \"kubernetes.io/projected/570ff1cb-b6dd-4045-859c-dc59335074f6-kube-api-access-msntq\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.768971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768756    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-tmp-dir\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.768971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768772    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/570ff1cb-b6dd-4045-859c-dc59335074f6-cabundle-cert\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.768971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768807    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-home\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.768971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768894    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-model-cache\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.768971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768967    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570ff1cb-b6dd-4045-859c-dc59335074f6-tls-certs\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.769239 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.768990    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-kserve-provision-location\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.769305 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.769260    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-tmp-dir\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.769421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.769384    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-model-cache\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.769518 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.769504    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-home\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.769641 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.769617    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-kserve-provision-location\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.769687 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.769663    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/570ff1cb-b6dd-4045-859c-dc59335074f6-cabundle-cert\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.771174 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.771125    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-dshm\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.771401 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.771383    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570ff1cb-b6dd-4045-859c-dc59335074f6-tls-certs\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.825565 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.825542    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msntq\" (UniqueName: \"kubernetes.io/projected/570ff1cb-b6dd-4045-859c-dc59335074f6-kube-api-access-msntq\") pod \"stop-feature-test-kserve-6df697898d-gs4hw\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:56.839004 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:56.838981    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:27:57.311900 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:57.311877    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"]
Oct 31 08:27:57.313901 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:27:57.313872    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570ff1cb_b6dd_4045_859c_dc59335074f6.slice/crio-5bf4cb26555522619695dee5a60db3f7b1255ad69ac6cb98031d95d6bd6dd02b WatchSource:0}: Error finding container 5bf4cb26555522619695dee5a60db3f7b1255ad69ac6cb98031d95d6bd6dd02b: Status 404 returned error can't find the container with id 5bf4cb26555522619695dee5a60db3f7b1255ad69ac6cb98031d95d6bd6dd02b
Oct 31 08:27:58.203842 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:58.203805    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" event={"ID":"570ff1cb-b6dd-4045-859c-dc59335074f6","Type":"ContainerStarted","Data":"1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8"}
Oct 31 08:27:58.204203 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:27:58.203875    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" event={"ID":"570ff1cb-b6dd-4045-859c-dc59335074f6","Type":"ContainerStarted","Data":"5bf4cb26555522619695dee5a60db3f7b1255ad69ac6cb98031d95d6bd6dd02b"}
Oct 31 08:28:00.213762 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:00.213730    4532 generic.go:356] "Generic (PLEG): container finished" podID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerID="1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8" exitCode=0
Oct 31 08:28:00.214134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:00.213774    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" event={"ID":"570ff1cb-b6dd-4045-859c-dc59335074f6","Type":"ContainerDied","Data":"1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8"}
Oct 31 08:28:01.219766 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:01.219732    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" event={"ID":"570ff1cb-b6dd-4045-859c-dc59335074f6","Type":"ContainerStarted","Data":"dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209"}
Oct 31 08:28:01.434729 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:01.434658    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podStartSLOduration=3.168156718 podStartE2EDuration="5.43463825s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.266481532s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:28:01.384409198 +0000 UTC m=+3531.886563013" watchObservedRunningTime="2026-10-31 08:28:01.43463825 +0000 UTC m=+3531.936792043"
Oct 31 08:28:01.839797 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:01.839763    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:28:03.811355 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:03.811305    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" containerID="cri-o://8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9" gracePeriod=45
Oct 31 08:28:04.039870 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.039825    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": dial tcp 10.133.0.51:8000: connect: connection refused"
Oct 31 08:28:04.666897 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.666872    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:28:04.839648 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.839614    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kserve-provision-location\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.839662    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tls-certs\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.839703    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-cabundle-cert\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.839753    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-model-cache\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.839902    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-home\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-home\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.839991    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kube-api-access-g7h8r\" (UniqueName: \"kubernetes.io/projected/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kube-api-access-g7h8r\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.840025    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-dshm\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.840034    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-model-cache" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.840061    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-cabundle-cert" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:28:04.840146 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.840135    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tmp-dir\") pod \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\" (UID: \"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb\") "
Oct 31 08:28:04.840650 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.840493    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:04.840650 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.840516    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:04.840650 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.840623    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-home" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:28:04.842275 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.842246    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tls-certs" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:28:04.842434 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.842405    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-dshm" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:28:04.842647 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.842621    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kube-api-access-g7h8r" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "kube-api-access-g7h8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:28:04.858195 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.858170    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tmp-dir" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:28:04.864224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.864200    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kserve-provision-location" pod "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" (UID: "3b4cf078-2a0c-4e85-a11c-2f012f4f49cb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:28:04.941920 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.941890    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:04.941920 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.941917    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:04.941920 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.941927    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:04.942134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.941937    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:04.942134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.941946    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:04.942134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:04.941954    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7h8r\" (UniqueName: \"kubernetes.io/projected/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb-kube-api-access-g7h8r\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:28:05.251320 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.251235    4532 generic.go:356] "Generic (PLEG): container finished" podID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerID="8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9" exitCode=0
Oct 31 08:28:05.251320 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.251294    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"
Oct 31 08:28:05.251539 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.251319    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" event={"ID":"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb","Type":"ContainerDied","Data":"8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9"}
Oct 31 08:28:05.251539 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.251378    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6" event={"ID":"3b4cf078-2a0c-4e85-a11c-2f012f4f49cb","Type":"ContainerDied","Data":"46d5eefd13d50e883a49543c03eabf30453aee8e9d6fdd2197df3b99ea0453b7"}
Oct 31 08:28:05.251539 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.251401    4532 scope.go:122] "RemoveContainer" containerID="8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9"
Oct 31 08:28:05.260029 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.260014    4532 scope.go:122] "RemoveContainer" containerID="a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838"
Oct 31 08:28:05.267627 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.267603    4532 scope.go:122] "RemoveContainer" containerID="8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9"
Oct 31 08:28:05.267899 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:28:05.267883    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9\": container with ID starting with 8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9 not found: ID does not exist" containerID="8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9"
Oct 31 08:28:05.267975 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.267910    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9"} err="failed to get container status \"8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9\": rpc error: code = NotFound desc = could not find container \"8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9\": container with ID starting with 8f06c0dca61ef93edac61580fed0e7a5d21ebce67cbf59a429c300c8ca3d0fe9 not found: ID does not exist"
Oct 31 08:28:05.267975 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.267926    4532 scope.go:122] "RemoveContainer" containerID="a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838"
Oct 31 08:28:05.268180 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:28:05.268164    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838\": container with ID starting with a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838 not found: ID does not exist" containerID="a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838"
Oct 31 08:28:05.268224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.268185    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838"} err="failed to get container status \"a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838\": rpc error: code = NotFound desc = could not find container \"a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838\": container with ID starting with a933372f4003bdf914253f77c0cf3fda233d2144f9e733cfbb5079b99eafe838 not found: ID does not exist"
Oct 31 08:28:05.415862 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.415825    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"]
Oct 31 08:28:05.464972 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.464940    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-d9kt6"]
Oct 31 08:28:05.630728 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:05.625436    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" path="/var/lib/kubelet/pods/3b4cf078-2a0c-4e85-a11c-2f012f4f49cb/volumes"
Oct 31 08:28:06.839407 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:06.839377    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:28:06.840877 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:06.840847    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:28:09.310771 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:28:09.310733    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:28:16.840358 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:16.840314    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:28:26.840026 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:26.839922    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:28:36.839833 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:36.839788    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:28:39.311065 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:28:39.311019    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:28:46.839978 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:46.839928    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:28:56.840177 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:28:56.840138    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:29:06.840228 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:06.840174    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:29:09.311876 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:29:09.311837    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:29:14.968661 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.968626    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"]
Oct 31 08:29:14.969151 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.969122    4532 cpu_manager.go:414] "containerMap: removing container" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main"
Oct 31 08:29:14.969151 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.969141    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main"
Oct 31 08:29:14.969230 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.969173    4532 cpu_manager.go:414] "containerMap: removing container" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="storage-initializer"
Oct 31 08:29:14.969230 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.969182    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="storage-initializer"
Oct 31 08:29:14.969324 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.969308    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b4cf078-2a0c-4e85-a11c-2f012f4f49cb" containerName="main"
Oct 31 08:29:14.972853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.972831    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:14.984498 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:14.984452    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"]
Oct 31 08:29:15.026926 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.026893    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"seaweedfs-s3-creds\""
Oct 31 08:29:15.027071 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.026983    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-zpxzg\""
Oct 31 08:29:15.031457 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.031436    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:29:15.031526 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.031447    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Oct 31 08:29:15.031611 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.031595    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Oct 31 08:29:15.031652 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.031633    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Oct 31 08:29:15.048069 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048044    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02171c7e-807a-4907-8e9a-4f2289dd5e68-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.048179 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048089    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.048179 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048161    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86vln\" (UniqueName: \"kubernetes.io/projected/02171c7e-807a-4907-8e9a-4f2289dd5e68-kube-api-access-86vln\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.048266 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048192    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02171c7e-807a-4907-8e9a-4f2289dd5e68-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.048308 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048286    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.048354 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048337    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.048396 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048369    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.048513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.048496    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.149731 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.149693    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.149893 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.149743    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86vln\" (UniqueName: \"kubernetes.io/projected/02171c7e-807a-4907-8e9a-4f2289dd5e68-kube-api-access-86vln\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.149893 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.149864    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02171c7e-807a-4907-8e9a-4f2289dd5e68-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.149984 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.149920    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.149984 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.149971    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150072 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150001    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150072 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150064    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150170 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150148    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02171c7e-807a-4907-8e9a-4f2289dd5e68-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150233 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150175    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150354 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150331    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150354 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150343    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150527 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150486    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.150659 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.150636    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02171c7e-807a-4907-8e9a-4f2289dd5e68-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.152443 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.152417    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.152867 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.152845    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02171c7e-807a-4907-8e9a-4f2289dd5e68-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.206363 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.206328    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86vln\" (UniqueName: \"kubernetes.io/projected/02171c7e-807a-4907-8e9a-4f2289dd5e68-kube-api-access-86vln\") pod \"custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.294384 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.294359    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:15.770085 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:15.769999    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"]
Oct 31 08:29:15.773660 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:29:15.773614    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02171c7e_807a_4907_8e9a_4f2289dd5e68.slice/crio-75cf93291c6a767dbb5606b35459904e67520bd0fb204ddb5757eb8f8b27cfb7 WatchSource:0}: Error finding container 75cf93291c6a767dbb5606b35459904e67520bd0fb204ddb5757eb8f8b27cfb7: Status 404 returned error can't find the container with id 75cf93291c6a767dbb5606b35459904e67520bd0fb204ddb5757eb8f8b27cfb7
Oct 31 08:29:16.553611 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:16.553569    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" event={"ID":"02171c7e-807a-4907-8e9a-4f2289dd5e68","Type":"ContainerStarted","Data":"30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481"}
Oct 31 08:29:16.554001 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:16.553654    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" event={"ID":"02171c7e-807a-4907-8e9a-4f2289dd5e68","Type":"ContainerStarted","Data":"75cf93291c6a767dbb5606b35459904e67520bd0fb204ddb5757eb8f8b27cfb7"}
Oct 31 08:29:16.840514 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:16.840414    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:29:18.564375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:18.564337    4532 generic.go:356] "Generic (PLEG): container finished" podID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerID="30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481" exitCode=0
Oct 31 08:29:18.564791 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:18.564412    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" event={"ID":"02171c7e-807a-4907-8e9a-4f2289dd5e68","Type":"ContainerDied","Data":"30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481"}
Oct 31 08:29:19.571806 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:19.571768    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" event={"ID":"02171c7e-807a-4907-8e9a-4f2289dd5e68","Type":"ContainerStarted","Data":"3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8"}
Oct 31 08:29:19.790370 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:19.790311    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podStartSLOduration=3.442015128 podStartE2EDuration="5.790297654s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.348282526s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:29:19.739560159 +0000 UTC m=+3610.241713989" watchObservedRunningTime="2026-10-31 08:29:19.790297654 +0000 UTC m=+3610.292451438"
Oct 31 08:29:25.295014 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:25.294968    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:25.295692 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:25.295053    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:29:25.296855 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:25.296820    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:29:26.839736 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:26.839691    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:29:35.295726 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:35.295674    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:29:36.839600 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:36.839554    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:29:39.312660 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:29:39.312622    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:29:45.295153 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:45.295100    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:29:46.839986 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:46.839940    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": dial tcp 10.133.0.52:8000: connect: connection refused"
Oct 31 08:29:55.295599 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:55.295560    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:29:56.849709 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:56.849675    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:29:56.857244 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:29:56.857218    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:30:03.976386 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:03.976040    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"]
Oct 31 08:30:05.295382 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:05.295341    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:30:09.312444 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:30:09.312404    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:30:15.295066 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:15.295025    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:30:18.999695 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:18.999622    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main" containerID="cri-o://dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209" gracePeriod=45
Oct 31 08:30:19.661340 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.661313    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:30:19.758681 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758600    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-kserve-provision-location\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.758681 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758646    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/570ff1cb-b6dd-4045-859c-dc59335074f6-kube-api-access-msntq\" (UniqueName: \"kubernetes.io/projected/570ff1cb-b6dd-4045-859c-dc59335074f6-kube-api-access-msntq\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.758890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758699    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-dshm\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.758890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758734    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-home\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-home\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.758890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758769    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-model-cache\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.758890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758798    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/570ff1cb-b6dd-4045-859c-dc59335074f6-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/570ff1cb-b6dd-4045-859c-dc59335074f6-cabundle-cert\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.759106 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758897    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/570ff1cb-b6dd-4045-859c-dc59335074f6-tls-certs\" (UniqueName: \"kubernetes.io/secret/570ff1cb-b6dd-4045-859c-dc59335074f6-tls-certs\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.759106 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.758925    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-tmp-dir\") pod \"570ff1cb-b6dd-4045-859c-dc59335074f6\" (UID: \"570ff1cb-b6dd-4045-859c-dc59335074f6\") "
Oct 31 08:30:19.759459 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.759428    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-model-cache" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:30:19.759699 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.759667    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570ff1cb-b6dd-4045-859c-dc59335074f6-cabundle-cert" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:30:19.759843 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.759733    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-home" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:30:19.761453 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.761408    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-dshm" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:30:19.761586 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.761503    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570ff1cb-b6dd-4045-859c-dc59335074f6-tls-certs" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:30:19.761739 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.761719    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570ff1cb-b6dd-4045-859c-dc59335074f6-kube-api-access-msntq" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "kube-api-access-msntq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:30:19.778443 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.778414    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-tmp-dir" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:30:19.785967 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.785940    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-kserve-provision-location" pod "570ff1cb-b6dd-4045-859c-dc59335074f6" (UID: "570ff1cb-b6dd-4045-859c-dc59335074f6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:30:19.843762 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.843728    4532 generic.go:356] "Generic (PLEG): container finished" podID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerID="dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209" exitCode=0
Oct 31 08:30:19.843933 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.843770    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" event={"ID":"570ff1cb-b6dd-4045-859c-dc59335074f6","Type":"ContainerDied","Data":"dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209"}
Oct 31 08:30:19.843933 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.843782    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"
Oct 31 08:30:19.843933 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.843820    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw" event={"ID":"570ff1cb-b6dd-4045-859c-dc59335074f6","Type":"ContainerDied","Data":"5bf4cb26555522619695dee5a60db3f7b1255ad69ac6cb98031d95d6bd6dd02b"}
Oct 31 08:30:19.843933 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.843849    4532 scope.go:122] "RemoveContainer" containerID="dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209"
Oct 31 08:30:19.852509 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.852491    4532 scope.go:122] "RemoveContainer" containerID="1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8"
Oct 31 08:30:19.859747 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859722    4532 scope.go:122] "RemoveContainer" containerID="dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209"
Oct 31 08:30:19.859950 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859929    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570ff1cb-b6dd-4045-859c-dc59335074f6-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859952    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859961    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859971    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msntq\" (UniqueName: \"kubernetes.io/projected/570ff1cb-b6dd-4045-859c-dc59335074f6-kube-api-access-msntq\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859979    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859987    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.859996    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570ff1cb-b6dd-4045-859c-dc59335074f6-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.860005    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/570ff1cb-b6dd-4045-859c-dc59335074f6-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:30:19.860387 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:30:19.860032    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209\": container with ID starting with dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209 not found: ID does not exist" containerID="dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209"
Oct 31 08:30:19.860387 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.860057    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209"} err="failed to get container status \"dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209\": rpc error: code = NotFound desc = could not find container \"dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209\": container with ID starting with dcf48f523bf9b9834accedf281026496144a868bb661af667015786ef1dca209 not found: ID does not exist"
Oct 31 08:30:19.860387 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.860076    4532 scope.go:122] "RemoveContainer" containerID="1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8"
Oct 31 08:30:19.860387 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:30:19.860295    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8\": container with ID starting with 1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8 not found: ID does not exist" containerID="1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8"
Oct 31 08:30:19.860387 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:19.860319    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8"} err="failed to get container status \"1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8\": rpc error: code = NotFound desc = could not find container \"1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8\": container with ID starting with 1c9474b220bf91330c42a01f6ffba3f3d53879e3558bccb9f15a7af4b24485d8 not found: ID does not exist"
Oct 31 08:30:20.010234 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:20.010142    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"]
Oct 31 08:30:20.015318 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:20.015290    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-6df697898d-gs4hw"]
Oct 31 08:30:21.625636 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:21.625604    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" path="/var/lib/kubelet/pods/570ff1cb-b6dd-4045-859c-dc59335074f6/volumes"
Oct 31 08:30:25.295005 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:25.294966    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:30:35.295610 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:35.295572    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:30:39.311093 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:30:39.311044    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:30:45.295008 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:45.294968    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:30:55.295457 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:30:55.295410    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:31:05.295498 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:05.295442    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": dial tcp 10.133.0.53:8000: connect: connection refused"
Oct 31 08:31:09.311306 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:31:09.311274    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:31:15.305276 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:15.305245    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:31:15.313273 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:15.313246    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:31:29.130851 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:29.130766    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"]
Oct 31 08:31:39.310990 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:31:39.310953    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:31:44.155603 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.155554    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main" containerID="cri-o://3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8" gracePeriod=45
Oct 31 08:31:44.796517 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.796489    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:31:44.860936 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.860907    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02171c7e-807a-4907-8e9a-4f2289dd5e68-kube-api-access-86vln\" (UniqueName: \"kubernetes.io/projected/02171c7e-807a-4907-8e9a-4f2289dd5e68-kube-api-access-86vln\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861087 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.860955    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-kserve-provision-location\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861087 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.861008    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-dshm\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-dshm\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861087 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.861049    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-home\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-home\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861249 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.861140    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-tmp-dir\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861249 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.861179    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02171c7e-807a-4907-8e9a-4f2289dd5e68-tls-certs\" (UniqueName: \"kubernetes.io/secret/02171c7e-807a-4907-8e9a-4f2289dd5e68-tls-certs\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861249 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.861204    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02171c7e-807a-4907-8e9a-4f2289dd5e68-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02171c7e-807a-4907-8e9a-4f2289dd5e68-cabundle-cert\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861249 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.861234    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-model-cache\") pod \"02171c7e-807a-4907-8e9a-4f2289dd5e68\" (UID: \"02171c7e-807a-4907-8e9a-4f2289dd5e68\") "
Oct 31 08:31:44.861852 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.861809    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02171c7e-807a-4907-8e9a-4f2289dd5e68-cabundle-cert" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:31:44.862183 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.862150    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-home" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:31:44.862183 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.862171    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-model-cache" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:31:44.863653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.863621    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-dshm" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:31:44.863653 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.863629    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02171c7e-807a-4907-8e9a-4f2289dd5e68-kube-api-access-86vln" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "kube-api-access-86vln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:31:44.864582 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.864554    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02171c7e-807a-4907-8e9a-4f2289dd5e68-tls-certs" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:31:44.875444 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.875417    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-tmp-dir" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:31:44.882457 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.882436    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-kserve-provision-location" pod "02171c7e-807a-4907-8e9a-4f2289dd5e68" (UID: "02171c7e-807a-4907-8e9a-4f2289dd5e68"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:31:44.962342 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962292    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:44.962342 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962312    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:44.962342 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962321    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02171c7e-807a-4907-8e9a-4f2289dd5e68-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:44.962342 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962330    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/02171c7e-807a-4907-8e9a-4f2289dd5e68-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:44.962342 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962339    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:44.962592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962348    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86vln\" (UniqueName: \"kubernetes.io/projected/02171c7e-807a-4907-8e9a-4f2289dd5e68-kube-api-access-86vln\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:44.962592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962357    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:44.962592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:44.962365    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02171c7e-807a-4907-8e9a-4f2289dd5e68-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:31:45.196010 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.195982    4532 generic.go:356] "Generic (PLEG): container finished" podID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerID="3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8" exitCode=0
Oct 31 08:31:45.196530 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.196043    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"
Oct 31 08:31:45.196530 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.196045    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" event={"ID":"02171c7e-807a-4907-8e9a-4f2289dd5e68","Type":"ContainerDied","Data":"3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8"}
Oct 31 08:31:45.196530 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.196160    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq" event={"ID":"02171c7e-807a-4907-8e9a-4f2289dd5e68","Type":"ContainerDied","Data":"75cf93291c6a767dbb5606b35459904e67520bd0fb204ddb5757eb8f8b27cfb7"}
Oct 31 08:31:45.196530 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.196182    4532 scope.go:122] "RemoveContainer" containerID="3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8"
Oct 31 08:31:45.205498 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.205456    4532 scope.go:122] "RemoveContainer" containerID="30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481"
Oct 31 08:31:45.212715 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.212698    4532 scope.go:122] "RemoveContainer" containerID="3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8"
Oct 31 08:31:45.212984 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:31:45.212967    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8\": container with ID starting with 3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8 not found: ID does not exist" containerID="3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8"
Oct 31 08:31:45.213027 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.212994    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8"} err="failed to get container status \"3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8\": rpc error: code = NotFound desc = could not find container \"3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8\": container with ID starting with 3094f0df1507c28b1181c59aa9c5d2c214ab6caad633a8229012e22c6ec353b8 not found: ID does not exist"
Oct 31 08:31:45.213027 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.213012    4532 scope.go:122] "RemoveContainer" containerID="30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481"
Oct 31 08:31:45.213264 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:31:45.213247    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481\": container with ID starting with 30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481 not found: ID does not exist" containerID="30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481"
Oct 31 08:31:45.213318 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.213267    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481"} err="failed to get container status \"30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481\": rpc error: code = NotFound desc = could not find container \"30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481\": container with ID starting with 30eb8b82198ebf8b81998fd6d3a3af1fbdf585f8a31663bc9e6f99f585d28481 not found: ID does not exist"
Oct 31 08:31:45.361308 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.361273    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"]
Oct 31 08:31:45.367368 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.367334    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-5f9bb68559-zjksq"]
Oct 31 08:31:45.625584 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:31:45.625559    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" path="/var/lib/kubelet/pods/02171c7e-807a-4907-8e9a-4f2289dd5e68/volumes"
Oct 31 08:32:09.310794 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:32:09.310757    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:32:33.907607 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.907567    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"]
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908022    4532 cpu_manager.go:414] "containerMap: removing container" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="storage-initializer"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908041    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="storage-initializer"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908071    4532 cpu_manager.go:414] "containerMap: removing container" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908079    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908095    4532 cpu_manager.go:414] "containerMap: removing container" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908102    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908120    4532 cpu_manager.go:414] "containerMap: removing container" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="storage-initializer"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908127    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="storage-initializer"
Oct 31 08:32:33.908210 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908211    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="02171c7e-807a-4907-8e9a-4f2289dd5e68" containerName="main"
Oct 31 08:32:33.908689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.908224    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="570ff1cb-b6dd-4045-859c-dc59335074f6" containerName="main"
Oct 31 08:32:33.912498 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.912475    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:33.961390 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:33.961358    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"]
Oct 31 08:32:34.009138 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.009110    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"seaweedfs-s3-creds\""
Oct 31 08:32:34.009273 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.009158    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-dockercfg-jwf4n\""
Oct 31 08:32:34.009273 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.009174    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Oct 31 08:32:34.009581 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.009561    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:32:34.009581 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.009572    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Oct 31 08:32:34.009581 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.009579    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Oct 31 08:32:34.009738 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.009601    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-5zqt7\""
Oct 31 08:32:34.032062 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032038    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-dshm\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.032183 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032078    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-home\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.032183 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032097    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/785d5684-3557-4657-b38f-593ac8ffb917-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.032310 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032226    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-tmp-dir\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.032310 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032257    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-model-cache\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.032406 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032344    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.032455 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032401    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d5684-3557-4657-b38f-593ac8ffb917-tls-certs\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.032455 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.032435    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxsb6\" (UniqueName: \"kubernetes.io/projected/785d5684-3557-4657-b38f-593ac8ffb917-kube-api-access-dxsb6\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133532 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133496    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-dshm\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133698 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133545    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-home\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133698 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133564    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/785d5684-3557-4657-b38f-593ac8ffb917-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133781 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133691    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-tmp-dir\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133781 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133730    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-model-cache\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133879 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133786    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133879 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133825    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d5684-3557-4657-b38f-593ac8ffb917-tls-certs\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.133879 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.133867    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxsb6\" (UniqueName: \"kubernetes.io/projected/785d5684-3557-4657-b38f-593ac8ffb917-kube-api-access-dxsb6\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.134059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.134033    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-home\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.134306 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.134280    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/785d5684-3557-4657-b38f-593ac8ffb917-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.134306 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.134295    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.134478 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.134307    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-tmp-dir\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.134569 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.134548    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-model-cache\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.135926 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.135907    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-dshm\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.136271 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.136252    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d5684-3557-4657-b38f-593ac8ffb917-tls-certs\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.189867 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.189810    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxsb6\" (UniqueName: \"kubernetes.io/projected/785d5684-3557-4657-b38f-593ac8ffb917-kube-api-access-dxsb6\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.231299 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.231270    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:34.704399 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.704374    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"]
Oct 31 08:32:34.706779 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:32:34.706750    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785d5684_3557_4657_b38f_593ac8ffb917.slice/crio-bfbe93d4e487db22b61f659f13a0738b55fb331679ac7ec5c1f825922a772b07 WatchSource:0}: Error finding container bfbe93d4e487db22b61f659f13a0738b55fb331679ac7ec5c1f825922a772b07: Status 404 returned error can't find the container with id bfbe93d4e487db22b61f659f13a0738b55fb331679ac7ec5c1f825922a772b07
Oct 31 08:32:34.708821 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:34.708803    4532 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 08:32:35.393232 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:35.393196    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerStarted","Data":"bfbe93d4e487db22b61f659f13a0738b55fb331679ac7ec5c1f825922a772b07"}
Oct 31 08:32:37.402623 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:37.402547    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerStarted","Data":"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"}
Oct 31 08:32:37.403000 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:37.402682    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:38.408778 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:38.408738    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerStarted","Data":"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"}
Oct 31 08:32:39.323394 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:32:39.323352    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:32:40.419597 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:40.419558    4532 generic.go:356] "Generic (PLEG): container finished" podID="785d5684-3557-4657-b38f-593ac8ffb917" containerID="f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa" exitCode=0
Oct 31 08:32:40.419980 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:40.419608    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerDied","Data":"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"}
Oct 31 08:32:41.426191 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:41.426150    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerStarted","Data":"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"}
Oct 31 08:32:41.644857 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:41.644798    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podStartSLOduration=4.6659243759999995 podStartE2EDuration="8.644779707s" totalImagesPullingTime="1.715862261s" totalInitContainerRuntime="2.26299307s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:32:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:32:41.644366096 +0000 UTC m=+3812.146519899" watchObservedRunningTime="2026-10-31 08:32:41.644779707 +0000 UTC m=+3812.146933499"
Oct 31 08:32:44.232147 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:44.232111    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:44.232574 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:44.232167    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:32:44.233672 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:44.233644    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:32:54.232270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:54.232164    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:32:54.249585 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:32:54.249559    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:33:04.232552 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:04.232501    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:33:09.310005 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:33:09.309965    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:33:14.232625 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:14.232579    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:33:22.173925 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:22.173891    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"]
Oct 31 08:33:22.174427 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:22.174121    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy" containerID="cri-o://2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9" gracePeriod=30
Oct 31 08:33:22.224026 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:22.223996    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"]
Oct 31 08:33:22.224236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:22.224199    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy" containerID="cri-o://a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316" gracePeriod=30
Oct 31 08:33:22.225134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:22.225111    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"]
Oct 31 08:33:22.225297 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:22.225279    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" containerID="cri-o://5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82" gracePeriod=30
Oct 31 08:33:24.232360 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:24.232310    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:33:25.935577 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:25.935539    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:33:25.935944 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:25.935615    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:33:25.942027 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:25.942000    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:33:25.942143 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:25.942041    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:33:26.185128 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:26.185090    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 08:33:26.185297 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:26.185184    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:33:27.507264 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.507230    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:33:27.509699 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.509683    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:33:27.524590 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.524572    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:33:27.591555 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591525    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-envoy\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.591555 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591557    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-certs\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.591749 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591582    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-token\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-token\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.591749 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591609    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-credential-socket\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.591749 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591640    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-kube-api-access-dxsd5\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-kube-api-access-dxsd5\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.591749 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591671    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istiod-ca-cert\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.591749 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591704    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istiod-ca-cert\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.591749 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591730    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-socket\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591756    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-certs\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591782    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-credential-socket\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591813    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-socket\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591849    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-podinfo\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591850    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-certs" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591874    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-credential-socket" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591890    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-data\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591927    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-token\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-token\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591972    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-envoy\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.592059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.591997    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-data\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592072    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-certs" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592105    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-kube-api-access-v5b7j\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-kube-api-access-v5b7j\") pod \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\" (UID: \"1b8061b9-ac7a-49eb-8ade-958158f44d7c\") "
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592139    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-podinfo\") pod \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\" (UID: \"d9051cc9-6e30-4b4d-bf3f-a792713499c5\") "
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592279    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-socket" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592217    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istiod-ca-cert" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592448    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592484    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592450    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-credential-socket" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592499    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592511    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-socket" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592540    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592559    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.592592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592553    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-data" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.593276 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592728    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-data" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.593276 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.592797    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istiod-ca-cert" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:33:27.594326 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.594255    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-envoy" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.594446 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.594323    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-token" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:33:27.594744 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.594715    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-kube-api-access-v5b7j" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "kube-api-access-v5b7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:33:27.594945 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.594930    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-podinfo" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:33:27.595137 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.595115    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-envoy" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.595660 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.595640    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-podinfo" pod "1b8061b9-ac7a-49eb-8ade-958158f44d7c" (UID: "1b8061b9-ac7a-49eb-8ade-958158f44d7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:33:27.595972 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.595946    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-kube-api-access-dxsd5" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "kube-api-access-dxsd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:33:27.596075 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.596052    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-token" pod "d9051cc9-6e30-4b4d-bf3f-a792713499c5" (UID: "d9051cc9-6e30-4b4d-bf3f-a792713499c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:33:27.638668 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.637800    4532 generic.go:356] "Generic (PLEG): container finished" podID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerID="a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316" exitCode=0
Oct 31 08:33:27.638668 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.637866    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"
Oct 31 08:33:27.638668 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.637906    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" event={"ID":"d9051cc9-6e30-4b4d-bf3f-a792713499c5","Type":"ContainerDied","Data":"a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316"}
Oct 31 08:33:27.638668 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.637955    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs" event={"ID":"d9051cc9-6e30-4b4d-bf3f-a792713499c5","Type":"ContainerDied","Data":"2569c73e0c4245c988e588f2f58a09ad11dae67a58885e5f94f7851de4e190bd"}
Oct 31 08:33:27.638668 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.637986    4532 scope.go:122] "RemoveContainer" containerID="a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316"
Oct 31 08:33:27.640375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.639964    4532 generic.go:356] "Generic (PLEG): container finished" podID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerID="2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9" exitCode=0
Oct 31 08:33:27.640375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.640016    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"
Oct 31 08:33:27.640375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.640103    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" event={"ID":"1b8061b9-ac7a-49eb-8ade-958158f44d7c","Type":"ContainerDied","Data":"2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9"}
Oct 31 08:33:27.640375 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.640146    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv" event={"ID":"1b8061b9-ac7a-49eb-8ade-958158f44d7c","Type":"ContainerDied","Data":"779bdfb5606277e9e2c708e602ac2792d021228b4156337bfd7b045ec4a75be3"}
Oct 31 08:33:27.642349 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.642330    4532 generic.go:356] "Generic (PLEG): container finished" podID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerID="5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82" exitCode=0
Oct 31 08:33:27.642453 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.642375    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" event={"ID":"b2bce18f-6dd2-44f8-81b4-166bff8123a1","Type":"ContainerDied","Data":"5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82"}
Oct 31 08:33:27.642453 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.642412    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf" event={"ID":"b2bce18f-6dd2-44f8-81b4-166bff8123a1","Type":"ContainerDied","Data":"2af3c597289ead1dbe38993aab5a01d8e18ae1cf784982e83eff7a6a66c1ac1d"}
Oct 31 08:33:27.642613 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.642501    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"
Oct 31 08:33:27.648145 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.648125    4532 scope.go:122] "RemoveContainer" containerID="a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316"
Oct 31 08:33:27.648419 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:33:27.648401    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316\": container with ID starting with a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316 not found: ID does not exist" containerID="a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316"
Oct 31 08:33:27.648507 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.648427    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316"} err="failed to get container status \"a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316\": rpc error: code = NotFound desc = could not find container \"a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316\": container with ID starting with a6adf9b68738111bc22e1f304c737e71d753f2c1ebf37bae344935a07acab316 not found: ID does not exist"
Oct 31 08:33:27.648507 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.648445    4532 scope.go:122] "RemoveContainer" containerID="2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9"
Oct 31 08:33:27.656332 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.656316    4532 scope.go:122] "RemoveContainer" containerID="2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9"
Oct 31 08:33:27.656768 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:33:27.656750    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9\": container with ID starting with 2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9 not found: ID does not exist" containerID="2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9"
Oct 31 08:33:27.656832 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.656778    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9"} err="failed to get container status \"2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9\": rpc error: code = NotFound desc = could not find container \"2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9\": container with ID starting with 2f9770b43c7f71a1705ace13c323a3ba0d47ae4bcf8c73595ad77f63409edad9 not found: ID does not exist"
Oct 31 08:33:27.656832 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.656796    4532 scope.go:122] "RemoveContainer" containerID="5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82"
Oct 31 08:33:27.663732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.663699    4532 scope.go:122] "RemoveContainer" containerID="5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82"
Oct 31 08:33:27.664012 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:33:27.663992    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82\": container with ID starting with 5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82 not found: ID does not exist" containerID="5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82"
Oct 31 08:33:27.664088 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.664017    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82"} err="failed to get container status \"5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82\": rpc error: code = NotFound desc = could not find container \"5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82\": container with ID starting with 5fc8543357082789c0e3fe220041e21c73ecba6fe54c95bed28b8f486e6cdd82 not found: ID does not exist"
Oct 31 08:33:27.693198 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693172    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-certs\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693208    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-data\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693249    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-kube-api-access-kqz5x\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-kube-api-access-kqz5x\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693278    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-envoy\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693331    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-credential-socket\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693360    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-token\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-token\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693394    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-podinfo\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693452    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-socket\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693489    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-certs" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.693887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693508    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-data" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.693887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693518    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istiod-ca-cert\") pod \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\" (UID: \"b2bce18f-6dd2-44f8-81b4-166bff8123a1\") "
Oct 31 08:33:27.693887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693737    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-credential-socket" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.693887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693819    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istiod-ca-cert" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:33:27.693887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693825    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.693887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693859    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693953    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-socket" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693969    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.693987    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694002    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694015    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694031    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694046    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694059    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5b7j\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-kube-api-access-v5b7j\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694073    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694087    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694102    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b8061b9-ac7a-49eb-8ade-958158f44d7c-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694117    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxsd5\" (UniqueName: \"kubernetes.io/projected/d9051cc9-6e30-4b4d-bf3f-a792713499c5-kube-api-access-dxsd5\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694131    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9051cc9-6e30-4b4d-bf3f-a792713499c5-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694143    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.694217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.694159    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b8061b9-ac7a-49eb-8ade-958158f44d7c-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.695959 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.695929    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-token" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:33:27.695959 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.695934    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-kube-api-access-kqz5x" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "kube-api-access-kqz5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:33:27.696071 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.695937    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-envoy" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:33:27.696243 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.696229    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-podinfo" pod "b2bce18f-6dd2-44f8-81b4-166bff8123a1" (UID: "b2bce18f-6dd2-44f8-81b4-166bff8123a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:33:27.795378 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.795349    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.795378 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.795378    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqz5x\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-kube-api-access-kqz5x\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.795592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.795392    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.795592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.795403    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.795592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.795415    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2bce18f-6dd2-44f8-81b4-166bff8123a1-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.795592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.795427    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2bce18f-6dd2-44f8-81b4-166bff8123a1-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:33:27.833733 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.833700    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"]
Oct 31 08:33:27.838789 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:27.838766    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-mlgzs"]
Oct 31 08:33:28.128378 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:28.128287    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"]
Oct 31 08:33:28.130941 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:28.130911    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-vmmpv"]
Oct 31 08:33:28.239740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:28.239708    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"]
Oct 31 08:33:28.243865 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:28.243837    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-26jsf"]
Oct 31 08:33:29.623646 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:29.623611    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" path="/var/lib/kubelet/pods/1b8061b9-ac7a-49eb-8ade-958158f44d7c/volumes"
Oct 31 08:33:29.624095 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:29.624079    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" path="/var/lib/kubelet/pods/b2bce18f-6dd2-44f8-81b4-166bff8123a1/volumes"
Oct 31 08:33:29.624522 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:29.624508    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" path="/var/lib/kubelet/pods/d9051cc9-6e30-4b4d-bf3f-a792713499c5/volumes"
Oct 31 08:33:34.232015 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:34.231970    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:33:39.313383 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:33:39.313349    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:33:44.232313 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:44.232254    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:33:54.232017 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:33:54.231972    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:34:04.232084 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:34:04.232030    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:34:09.311646 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:34:09.311615    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:34:14.231909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:34:14.231862    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:34:24.232077 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:34:24.231982    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8001/health\": dial tcp 10.133.0.54:8001: connect: connection refused"
Oct 31 08:34:34.241681 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:34:34.241649    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:34:34.254056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:34:34.254029    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:34:39.313359 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:34:39.313315    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:34:48.093224 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:34:48.093188    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"]
Oct 31 08:35:03.117160 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:03.117099    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main" containerID="cri-o://3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f" gracePeriod=45
Oct 31 08:35:03.733862 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:03.733813    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="llm-d-routing-sidecar" containerID="cri-o://c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56" gracePeriod=45
Oct 31 08:35:03.978376 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:03.978347    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:35:04.020923 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.020895    4532 generic.go:356] "Generic (PLEG): container finished" podID="785d5684-3557-4657-b38f-593ac8ffb917" containerID="3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f" exitCode=0
Oct 31 08:35:04.020923 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.020920    4532 generic.go:356] "Generic (PLEG): container finished" podID="785d5684-3557-4657-b38f-593ac8ffb917" containerID="c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56" exitCode=0
Oct 31 08:35:04.021086 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.020946    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"
Oct 31 08:35:04.021086 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.020980    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerDied","Data":"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"}
Oct 31 08:35:04.021086 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.021035    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerDied","Data":"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"}
Oct 31 08:35:04.021086 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.021052    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp" event={"ID":"785d5684-3557-4657-b38f-593ac8ffb917","Type":"ContainerDied","Data":"bfbe93d4e487db22b61f659f13a0738b55fb331679ac7ec5c1f825922a772b07"}
Oct 31 08:35:04.021221 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.021072    4532 scope.go:122] "RemoveContainer" containerID="3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"
Oct 31 08:35:04.034288 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.034274    4532 scope.go:122] "RemoveContainer" containerID="f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"
Oct 31 08:35:04.041248 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.041211    4532 scope.go:122] "RemoveContainer" containerID="c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"
Oct 31 08:35:04.048303 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.048284    4532 scope.go:122] "RemoveContainer" containerID="3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"
Oct 31 08:35:04.048546 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:35:04.048530    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f\": container with ID starting with 3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f not found: ID does not exist" containerID="3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"
Oct 31 08:35:04.048606 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.048556    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"} err="failed to get container status \"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f\": rpc error: code = NotFound desc = could not find container \"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f\": container with ID starting with 3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f not found: ID does not exist"
Oct 31 08:35:04.048606 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.048571    4532 scope.go:122] "RemoveContainer" containerID="f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"
Oct 31 08:35:04.048785 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:35:04.048768    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa\": container with ID starting with f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa not found: ID does not exist" containerID="f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"
Oct 31 08:35:04.048825 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.048791    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"} err="failed to get container status \"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa\": rpc error: code = NotFound desc = could not find container \"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa\": container with ID starting with f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa not found: ID does not exist"
Oct 31 08:35:04.048825 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.048807    4532 scope.go:122] "RemoveContainer" containerID="c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"
Oct 31 08:35:04.049057 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:35:04.049042    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56\": container with ID starting with c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56 not found: ID does not exist" containerID="c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"
Oct 31 08:35:04.049113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.049059    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"} err="failed to get container status \"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56\": rpc error: code = NotFound desc = could not find container \"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56\": container with ID starting with c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56 not found: ID does not exist"
Oct 31 08:35:04.049113 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.049070    4532 scope.go:122] "RemoveContainer" containerID="3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"
Oct 31 08:35:04.049275 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.049260    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f"} err="failed to get container status \"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f\": rpc error: code = NotFound desc = could not find container \"3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f\": container with ID starting with 3fd29c341ec83913eff7a09e9e9b0d37a93c47412d7e8131a071f5fb3c1e8c6f not found: ID does not exist"
Oct 31 08:35:04.049321 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.049275    4532 scope.go:122] "RemoveContainer" containerID="f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"
Oct 31 08:35:04.049541 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.049524    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa"} err="failed to get container status \"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa\": rpc error: code = NotFound desc = could not find container \"f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa\": container with ID starting with f88859f8a80ea2a2d673e9a3987ad5482adcde7c5a304c10b7cd3eae014da4fa not found: ID does not exist"
Oct 31 08:35:04.049541 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.049541    4532 scope.go:122] "RemoveContainer" containerID="c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"
Oct 31 08:35:04.049787 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.049772    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56"} err="failed to get container status \"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56\": rpc error: code = NotFound desc = could not find container \"c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56\": container with ID starting with c4dfc3e05f46893c4ef811d21dd89a796a614afc92f19393c52da3a2ad77cb56 not found: ID does not exist"
Oct 31 08:35:04.070409 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070389    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-kserve-provision-location\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.070532 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070427    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-tmp-dir\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.070532 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070490    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-dshm\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.070532 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070523    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/785d5684-3557-4657-b38f-593ac8ffb917-tls-certs\" (UniqueName: \"kubernetes.io/secret/785d5684-3557-4657-b38f-593ac8ffb917-tls-certs\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.070685 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070556    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-home\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-home\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.070685 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070625    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/785d5684-3557-4657-b38f-593ac8ffb917-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/785d5684-3557-4657-b38f-593ac8ffb917-cabundle-cert\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.070685 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070670    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-model-cache\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.070834 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.070704    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/785d5684-3557-4657-b38f-593ac8ffb917-kube-api-access-dxsb6\" (UniqueName: \"kubernetes.io/projected/785d5684-3557-4657-b38f-593ac8ffb917-kube-api-access-dxsb6\") pod \"785d5684-3557-4657-b38f-593ac8ffb917\" (UID: \"785d5684-3557-4657-b38f-593ac8ffb917\") "
Oct 31 08:35:04.071278 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.071076    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/785d5684-3557-4657-b38f-593ac8ffb917-cabundle-cert" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:35:04.071278 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.071189    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-model-cache" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:35:04.071437 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.071327    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-home" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:35:04.072712 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.072687    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-dshm" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:35:04.073125 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.073090    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/785d5684-3557-4657-b38f-593ac8ffb917-tls-certs" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:35:04.073209 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.073173    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785d5684-3557-4657-b38f-593ac8ffb917-kube-api-access-dxsb6" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "kube-api-access-dxsb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:35:04.087415 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.087370    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-tmp-dir" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:35:04.095282 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.095261    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-kserve-provision-location" pod "785d5684-3557-4657-b38f-593ac8ffb917" (UID: "785d5684-3557-4657-b38f-593ac8ffb917"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:35:04.171637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171588    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.171637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171607    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/785d5684-3557-4657-b38f-593ac8ffb917-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.171637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171616    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.171637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171626    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxsb6\" (UniqueName: \"kubernetes.io/projected/785d5684-3557-4657-b38f-593ac8ffb917-kube-api-access-dxsb6\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.171637 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171634    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.172043 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171643    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.172043 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171652    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/785d5684-3557-4657-b38f-593ac8ffb917-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.172043 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.171660    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/785d5684-3557-4657-b38f-593ac8ffb917-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:35:04.442388 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.442327    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"]
Oct 31 08:35:04.446405 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:04.446381    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-fwmpp"]
Oct 31 08:35:05.623731 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:05.623700    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="785d5684-3557-4657-b38f-593ac8ffb917" path="/var/lib/kubelet/pods/785d5684-3557-4657-b38f-593ac8ffb917/volumes"
Oct 31 08:35:09.310413 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:35:09.310378    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:35:39.311120 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:35:39.311075    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:35:42.657050 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657020    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"]
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657296    4532 cpu_manager.go:414] "containerMap: removing container" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657306    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657321    4532 cpu_manager.go:414] "containerMap: removing container" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="llm-d-routing-sidecar"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657326    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="llm-d-routing-sidecar"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657334    4532 cpu_manager.go:414] "containerMap: removing container" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657339    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657348    4532 cpu_manager.go:414] "containerMap: removing container" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657352    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657358    4532 cpu_manager.go:414] "containerMap: removing container" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="storage-initializer"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657362    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="storage-initializer"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657374    4532 cpu_manager.go:414] "containerMap: removing container" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657378    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657429    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="llm-d-routing-sidecar"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657437    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="785d5684-3557-4657-b38f-593ac8ffb917" containerName="main"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657443    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9051cc9-6e30-4b4d-bf3f-a792713499c5" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657450    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b8061b9-ac7a-49eb-8ade-958158f44d7c" containerName="istio-proxy"
Oct 31 08:35:42.657513 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.657457    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2bce18f-6dd2-44f8-81b4-166bff8123a1" containerName="istio-proxy"
Oct 31 08:35:42.660443 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.660427    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.713037 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.713014    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"]
Oct 31 08:35:42.735781 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.735757    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.735890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.735797    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-dshm\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.735890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.735817    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.735890 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.735865    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d989x\" (UniqueName: \"kubernetes.io/projected/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kube-api-access-d989x\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.735997 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.735949    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-home\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.735997 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.735970    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.736060 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.736000    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.736060 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.736042    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.761660 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.761637    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"kube-root-ca.crt\""
Oct 31 08:35:42.761772 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.761663    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:35:42.761772 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.761684    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"seaweedfs-s3-creds\""
Oct 31 08:35:42.761772 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.761744    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs\""
Oct 31 08:35:42.761896 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.761778    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"default-dockercfg-qwlxk\""
Oct 31 08:35:42.761945 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.761912    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-fac1c4ae\"/\"openshift-service-ca.crt\""
Oct 31 08:35:42.836643 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836610    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-home\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.836761 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836651    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.836826 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836766    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.836826 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836794    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.836939 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836828    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.836939 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836884    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-dshm\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.836939 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836913    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.837096 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.836942    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d989x\" (UniqueName: \"kubernetes.io/projected/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kube-api-access-d989x\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.837096 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.837040    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-home\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.837096 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.837088    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.837275 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.837251    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.837337 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.837322    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.837479 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.837442    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.838967 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.838949    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-dshm\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.839236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.839219    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.893223 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.893203    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d989x\" (UniqueName: \"kubernetes.io/projected/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kube-api-access-d989x\") pod \"llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") " pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:42.974576 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:42.974517    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:43.442717 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:43.442689    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"]
Oct 31 08:35:43.446009 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:35:43.445975    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87fbfd25_1fbd_4394_b585_a4a1789ba9cb.slice/crio-a5c14feb18895c1ce4ef9f700714d6718eaf3b449cc131bc327d138d2b534c2c WatchSource:0}: Error finding container a5c14feb18895c1ce4ef9f700714d6718eaf3b449cc131bc327d138d2b534c2c: Status 404 returned error can't find the container with id a5c14feb18895c1ce4ef9f700714d6718eaf3b449cc131bc327d138d2b534c2c
Oct 31 08:35:44.163666 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:44.163605    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" event={"ID":"87fbfd25-1fbd-4394-b585-a4a1789ba9cb","Type":"ContainerStarted","Data":"75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432"}
Oct 31 08:35:44.164071 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:44.163687    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" event={"ID":"87fbfd25-1fbd-4394-b585-a4a1789ba9cb","Type":"ContainerStarted","Data":"a5c14feb18895c1ce4ef9f700714d6718eaf3b449cc131bc327d138d2b534c2c"}
Oct 31 08:35:46.173203 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:46.173169    4532 generic.go:356] "Generic (PLEG): container finished" podID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerID="75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432" exitCode=0
Oct 31 08:35:46.173592 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:46.173219    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" event={"ID":"87fbfd25-1fbd-4394-b585-a4a1789ba9cb","Type":"ContainerDied","Data":"75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432"}
Oct 31 08:35:47.178591 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:47.178557    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" event={"ID":"87fbfd25-1fbd-4394-b585-a4a1789ba9cb","Type":"ContainerStarted","Data":"52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4"}
Oct 31 08:35:47.346706 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:47.346659    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podStartSLOduration=3.096970447 podStartE2EDuration="5.34664574s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.249675293s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:35:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:35:47.345319514 +0000 UTC m=+3997.847473303" watchObservedRunningTime="2026-10-31 08:35:47.34664574 +0000 UTC m=+3997.848799529"
Oct 31 08:35:47.975049 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:47.975016    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:52.975553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:52.975459    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:35:52.977173 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:35:52.977142    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:36:02.974938 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:36:02.974895    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:36:09.312760 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:36:09.312718    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:36:12.975787 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:36:12.975746    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:36:22.974887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:36:22.974844    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:36:32.975527 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:36:32.975487    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:36:39.313910 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:36:39.313875    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:36:42.974874 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:36:42.974834    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:36:52.975840 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:36:52.975794    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:37:02.975560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:02.975511    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:37:09.311541 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:37:09.311418    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:37:12.975689 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:12.975643    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:37:22.975857 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:22.975766    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" probeResult="failure" output="Get \"https://10.133.0.55:8000/health\": dial tcp 10.133.0.55:8000: connect: connection refused"
Oct 31 08:37:32.985126 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:32.985085    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:37:32.992834 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:32.992812    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:37:39.326925 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:37:39.326883    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:37:41.089930 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:41.089894    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"]
Oct 31 08:37:56.112150 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.112082    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main" containerID="cri-o://52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4" gracePeriod=45
Oct 31 08:37:56.854544 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.854518    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:37:56.917680 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917620    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-dshm\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.917680 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917659    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-home\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-home\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.917857 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917727    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kube-api-access-d989x\" (UniqueName: \"kubernetes.io/projected/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kube-api-access-d989x\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.917857 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917757    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tmp-dir\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.917857 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917791    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-model-cache\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.917857 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917829    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kserve-provision-location\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.918060 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917880    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-cabundle-cert\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.918060 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.917947    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tls-certs\" (UniqueName: \"kubernetes.io/secret/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tls-certs\") pod \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\" (UID: \"87fbfd25-1fbd-4394-b585-a4a1789ba9cb\") "
Oct 31 08:37:56.918156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.918057    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-model-cache" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:37:56.918269 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.918246    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:56.918515 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.918490    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-home" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:37:56.918662 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.918640    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-cabundle-cert" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:37:56.920025 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.919999    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kube-api-access-d989x" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "kube-api-access-d989x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:37:56.920132 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.920089    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-dshm" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:37:56.920132 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.920102    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tls-certs" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:37:56.934914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.934878    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tmp-dir" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:37:56.948063 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:56.948041    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kserve-provision-location" pod "87fbfd25-1fbd-4394-b585-a4a1789ba9cb" (UID: "87fbfd25-1fbd-4394-b585-a4a1789ba9cb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:37:57.018746 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.018723    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:57.018746 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.018743    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:57.018866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.018753    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d989x\" (UniqueName: \"kubernetes.io/projected/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kube-api-access-d989x\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:57.018866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.018782    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:57.018866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.018793    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:57.018866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.018801    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:57.018866 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.018810    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87fbfd25-1fbd-4394-b585-a4a1789ba9cb-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:37:57.700700 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.700664    4532 generic.go:356] "Generic (PLEG): container finished" podID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerID="52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4" exitCode=0
Oct 31 08:37:57.701075 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.700715    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"
Oct 31 08:37:57.701075 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.700740    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" event={"ID":"87fbfd25-1fbd-4394-b585-a4a1789ba9cb","Type":"ContainerDied","Data":"52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4"}
Oct 31 08:37:57.701075 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.700779    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb" event={"ID":"87fbfd25-1fbd-4394-b585-a4a1789ba9cb","Type":"ContainerDied","Data":"a5c14feb18895c1ce4ef9f700714d6718eaf3b449cc131bc327d138d2b534c2c"}
Oct 31 08:37:57.701075 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.700801    4532 scope.go:122] "RemoveContainer" containerID="52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4"
Oct 31 08:37:57.708418 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.708399    4532 scope.go:122] "RemoveContainer" containerID="75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432"
Oct 31 08:37:57.717156 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.717092    4532 scope.go:122] "RemoveContainer" containerID="52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4"
Oct 31 08:37:57.717608 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:37:57.717578    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4\": container with ID starting with 52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4 not found: ID does not exist" containerID="52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4"
Oct 31 08:37:57.717702 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.717616    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4"} err="failed to get container status \"52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4\": rpc error: code = NotFound desc = could not find container \"52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4\": container with ID starting with 52109c164d994fc1fa13fafdd7bd03fe4f4e20150a3f2a7139433734c5d30db4 not found: ID does not exist"
Oct 31 08:37:57.717702 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.717636    4532 scope.go:122] "RemoveContainer" containerID="75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432"
Oct 31 08:37:57.717939 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:37:57.717921    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432\": container with ID starting with 75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432 not found: ID does not exist" containerID="75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432"
Oct 31 08:37:57.717973 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.717946    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432"} err="failed to get container status \"75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432\": rpc error: code = NotFound desc = could not find container \"75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432\": container with ID starting with 75082bfb3b34b99565426e647339ebae91960c64ed4e1011e96b53e9a508a432 not found: ID does not exist"
Oct 31 08:37:57.915102 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.915045    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"]
Oct 31 08:37:57.964903 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:57.964880    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-fac1c4ae/llmisvc-model-fb-opt-125m-route-3c960099-kserve-66684585447cfrb"]
Oct 31 08:37:59.624330 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:37:59.624295    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" path="/var/lib/kubelet/pods/87fbfd25-1fbd-4394-b585-a4a1789ba9cb/volumes"
Oct 31 08:38:09.310032 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:38:09.310000    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:38:10.591565 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.591533    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 31 08:38:10.591963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.591881    4532 cpu_manager.go:414] "containerMap: removing container" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main"
Oct 31 08:38:10.591963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.591896    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main"
Oct 31 08:38:10.591963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.591913    4532 cpu_manager.go:414] "containerMap: removing container" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="storage-initializer"
Oct 31 08:38:10.591963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.591919    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="storage-initializer"
Oct 31 08:38:10.592098 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.591969    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="87fbfd25-1fbd-4394-b585-a4a1789ba9cb" containerName="main"
Oct 31 08:38:10.596783 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.596767    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.650350 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.650324    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-x5bhj\""
Oct 31 08:38:10.650350 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.650349    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:38:10.650553 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.650363    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"openshift-service-ca.crt\""
Oct 31 08:38:10.653100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.653087    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"seaweedfs-s3-creds\""
Oct 31 08:38:10.701227 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.701197    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 31 08:38:10.726606 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726578    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.726733 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726639    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.726733 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726715    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fskl8\" (UniqueName: \"kubernetes.io/projected/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kube-api-access-fskl8\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.726805 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726740    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.726805 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726758    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.726805 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726781    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.726805 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726795    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.726929 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.726842    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.750760 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.750736    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Oct 31 08:38:10.750871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.750793    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"default-dockercfg-khhlv\""
Oct 31 08:38:10.750871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.750809    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"kube-root-ca.crt\""
Oct 31 08:38:10.827514 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827486    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827514 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827517    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827544    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827569    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827587    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827610    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827900 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827721    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827900 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827776    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fskl8\" (UniqueName: \"kubernetes.io/projected/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kube-api-access-fskl8\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827999 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827922    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.827999 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.827956    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.828102 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.828001    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.828102 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.828049    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.828388 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.828369    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.829868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.829846    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.829972 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.829896    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.884996 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.884933    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fskl8\" (UniqueName: \"kubernetes.io/projected/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kube-api-access-fskl8\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:10.911889 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:10.911860    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:11.382531 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:11.382390    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 31 08:38:11.385198 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:38:11.385165    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37412c2e_8fc7_404d_b5f0_6d6e04bd32c2.slice/crio-8091bade8c59cf7e1ffbfe41a2aa52e7c1c3015e5088dffc1b4bf216268482df WatchSource:0}: Error finding container 8091bade8c59cf7e1ffbfe41a2aa52e7c1c3015e5088dffc1b4bf216268482df: Status 404 returned error can't find the container with id 8091bade8c59cf7e1ffbfe41a2aa52e7c1c3015e5088dffc1b4bf216268482df
Oct 31 08:38:11.753184 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:11.753108    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2","Type":"ContainerStarted","Data":"d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0"}
Oct 31 08:38:11.753184 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:11.753159    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2","Type":"ContainerStarted","Data":"8091bade8c59cf7e1ffbfe41a2aa52e7c1c3015e5088dffc1b4bf216268482df"}
Oct 31 08:38:13.762520 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:13.762487    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2","Type":"ContainerDied","Data":"d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0"}
Oct 31 08:38:13.762875 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:13.762489    4532 generic.go:356] "Generic (PLEG): container finished" podID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerID="d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0" exitCode=0
Oct 31 08:38:14.772694 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:14.772646    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2","Type":"ContainerStarted","Data":"c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc"}
Oct 31 08:38:14.912832 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:14.912800    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:14.992069 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:14.992015    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podStartSLOduration=2.780842356 podStartE2EDuration="4.992002374s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.211160018s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:38:14.941232258 +0000 UTC m=+4145.443386059" watchObservedRunningTime="2026-10-31 08:38:14.992002374 +0000 UTC m=+4145.494156163"
Oct 31 08:38:20.912665 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:20.912630    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:38:20.914214 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:20.914186    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:38:30.912673 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:30.912634    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:38:39.313682 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:38:39.313645    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:38:40.913121 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:40.913082    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:38:50.913068 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:38:50.912986    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:39:00.912699 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:39:00.912648    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:39:09.313591 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:39:09.313553    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:39:10.912301 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:39:10.912254    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:39:20.912879 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:39:20.912829    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:39:30.912732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:39:30.912682    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:39:39.319672 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:39:39.319638    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:39:40.912278 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:39:40.912237    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.56:8000/health\": dial tcp 10.133.0.56:8000: connect: connection refused"
Oct 31 08:39:50.921901 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:39:50.921867    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:39:50.929270 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:39:50.929242    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:40:00.960921 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:00.960876    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 31 08:40:09.311266 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:40:09.311228    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:40:15.984013 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:15.983974    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main" containerID="cri-o://c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc" gracePeriod=45
Oct 31 08:40:16.719459 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.719437    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:40:16.836610 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836588    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-cabundle-cert\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.836722 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836624    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tmp-dir\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.836722 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836645    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kube-api-access-fskl8\" (UniqueName: \"kubernetes.io/projected/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kube-api-access-fskl8\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.836722 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836683    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-home\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-home\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.836722 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836708    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tls-certs\" (UniqueName: \"kubernetes.io/secret/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tls-certs\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.836960 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836723    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-model-cache\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.836960 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836753    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kserve-provision-location\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.836960 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836785    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-dshm\") pod \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\" (UID: \"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2\") "
Oct 31 08:40:16.837118 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836982    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-cabundle-cert" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:40:16.837118 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.836990    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-model-cache" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:40:16.837329 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.837305    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-home" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:40:16.839214 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.839195    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-dshm" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:40:16.839344 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.839327    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kube-api-access-fskl8" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "kube-api-access-fskl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:40:16.839399 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.839368    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tls-certs" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:40:16.850398 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.850365    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tmp-dir" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:40:16.861650 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.861625    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kserve-provision-location" pod "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" (UID: "37412c2e-8fc7-404d-b5f0-6d6e04bd32c2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:40:16.937371 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937346    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:16.937371 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937368    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:16.937521 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937379    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:16.937521 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937387    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:16.937521 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937396    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fskl8\" (UniqueName: \"kubernetes.io/projected/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-kube-api-access-fskl8\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:16.937521 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937405    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:16.937521 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937413    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:16.937521 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:16.937421    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:40:17.241923 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.241859    4532 generic.go:356] "Generic (PLEG): container finished" podID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerID="c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc" exitCode=0
Oct 31 08:40:17.241923 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.241909    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 31 08:40:17.241923 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.241918    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2","Type":"ContainerDied","Data":"c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc"}
Oct 31 08:40:17.242379 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.241960    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"37412c2e-8fc7-404d-b5f0-6d6e04bd32c2","Type":"ContainerDied","Data":"8091bade8c59cf7e1ffbfe41a2aa52e7c1c3015e5088dffc1b4bf216268482df"}
Oct 31 08:40:17.242379 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.241982    4532 scope.go:122] "RemoveContainer" containerID="c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc"
Oct 31 08:40:17.250423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.250408    4532 scope.go:122] "RemoveContainer" containerID="d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0"
Oct 31 08:40:17.257454 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.257437    4532 scope.go:122] "RemoveContainer" containerID="c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc"
Oct 31 08:40:17.257746 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:40:17.257726    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc\": container with ID starting with c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc not found: ID does not exist" containerID="c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc"
Oct 31 08:40:17.257800 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.257755    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc"} err="failed to get container status \"c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc\": rpc error: code = NotFound desc = could not find container \"c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc\": container with ID starting with c2b066f55348d6d0740a55656ce1053012119b8a31205e7e08854a18a9f4bddc not found: ID does not exist"
Oct 31 08:40:17.257800 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.257772    4532 scope.go:122] "RemoveContainer" containerID="d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0"
Oct 31 08:40:17.257990 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:40:17.257974    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0\": container with ID starting with d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0 not found: ID does not exist" containerID="d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0"
Oct 31 08:40:17.258038 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.257995    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0"} err="failed to get container status \"d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0\": rpc error: code = NotFound desc = could not find container \"d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0\": container with ID starting with d02e464d69925948391d4e74d0427ce5dfc70784a1f2b3a13b02e901ac4cf5e0 not found: ID does not exist"
Oct 31 08:40:17.405865 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.405833    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 31 08:40:17.455524 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.455458    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 31 08:40:17.623674 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:17.623642    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" path="/var/lib/kubelet/pods/37412c2e-8fc7-404d-b5f0-6d6e04bd32c2/volumes"
Oct 31 08:40:39.310737 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:40:39.310704    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:40:57.275421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.275381    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"]
Oct 31 08:40:57.275953 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.275820    4532 cpu_manager.go:414] "containerMap: removing container" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="storage-initializer"
Oct 31 08:40:57.275953 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.275837    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="storage-initializer"
Oct 31 08:40:57.275953 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.275880    4532 cpu_manager.go:414] "containerMap: removing container" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main"
Oct 31 08:40:57.275953 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.275889    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main"
Oct 31 08:40:57.276173 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.275955    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="37412c2e-8fc7-404d-b5f0-6d6e04bd32c2" containerName="main"
Oct 31 08:40:57.279053 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.279032    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.331973 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.331941    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:40:57.331973 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.331968    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"kube-root-ca.crt\""
Oct 31 08:40:57.332167 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.331979    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"openshift-service-ca.crt\""
Oct 31 08:40:57.332599 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.332581    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"seaweedfs-s3-creds\""
Oct 31 08:40:57.332691 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.332624    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"default-dockercfg-drb5d\""
Oct 31 08:40:57.381814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.381788    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"tls-verification-test-kserve-dockercfg-4grsz\""
Oct 31 08:40:57.381913 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.381896    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"tls-verification-test-kserve-self-signed-certs\""
Oct 31 08:40:57.391044 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.391023    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"]
Oct 31 08:40:57.417591 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417566    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-dshm\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.417705 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417619    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-tmp-dir\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.417772 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417718    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-home\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.417772 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417747    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3446fda8-34bc-4ed4-859a-582906a74eff-cabundle-cert\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.417862 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417830    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3446fda8-34bc-4ed4-859a-582906a74eff-tls-certs\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.417916 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417860    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj947\" (UniqueName: \"kubernetes.io/projected/3446fda8-34bc-4ed4-859a-582906a74eff-kube-api-access-gj947\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.417916 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417880    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-model-cache\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.418016 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.417997    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-kserve-provision-location\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518320 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518293    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3446fda8-34bc-4ed4-859a-582906a74eff-tls-certs\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518503 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518329    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj947\" (UniqueName: \"kubernetes.io/projected/3446fda8-34bc-4ed4-859a-582906a74eff-kube-api-access-gj947\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518503 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518353    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-model-cache\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518503 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518389    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-kserve-provision-location\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518503 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518407    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-dshm\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518503 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518441    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-tmp-dir\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518503 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518500    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-home\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518825 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518527    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3446fda8-34bc-4ed4-859a-582906a74eff-cabundle-cert\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518884 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518853    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-model-cache\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.518959 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.518935    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-kserve-provision-location\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.519184 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.519161    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-tmp-dir\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.519268 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.519192    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-home\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.519268 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.519229    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3446fda8-34bc-4ed4-859a-582906a74eff-cabundle-cert\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.520654 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.520633    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-dshm\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.520991 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.520971    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3446fda8-34bc-4ed4-859a-582906a74eff-tls-certs\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.578342 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.578283    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj947\" (UniqueName: \"kubernetes.io/projected/3446fda8-34bc-4ed4-859a-582906a74eff-kube-api-access-gj947\") pod \"tls-verification-test-kserve-656f77bc69-2wwtb\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:57.595152 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:57.595132    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:58.020155 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:58.020080    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"]
Oct 31 08:40:58.024357 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:40:58.024327    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3446fda8_34bc_4ed4_859a_582906a74eff.slice/crio-aa89f6db95156feefda157740374cf319ec1a650d258b44eb4bd0e498e6ce78d WatchSource:0}: Error finding container aa89f6db95156feefda157740374cf319ec1a650d258b44eb4bd0e498e6ce78d: Status 404 returned error can't find the container with id aa89f6db95156feefda157740374cf319ec1a650d258b44eb4bd0e498e6ce78d
Oct 31 08:40:58.393293 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:58.393259    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerStarted","Data":"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"}
Oct 31 08:40:58.393787 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:58.393307    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerStarted","Data":"aa89f6db95156feefda157740374cf319ec1a650d258b44eb4bd0e498e6ce78d"}
Oct 31 08:40:58.393787 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:58.393342    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:40:59.398616 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:40:59.398568    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerStarted","Data":"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"}
Oct 31 08:41:01.410490 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:01.410445    4532 generic.go:356] "Generic (PLEG): container finished" podID="3446fda8-34bc-4ed4-859a-582906a74eff" containerID="79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3" exitCode=0
Oct 31 08:41:01.410879 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:01.410505    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerDied","Data":"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"}
Oct 31 08:41:02.416873 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:02.416836    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerStarted","Data":"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"}
Oct 31 08:41:02.643784 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:02.643701    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podStartSLOduration=3.480624025 podStartE2EDuration="5.64368329s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.163059265s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:41:02.642148735 +0000 UTC m=+4313.144302538" watchObservedRunningTime="2026-10-31 08:41:02.64368329 +0000 UTC m=+4313.145837079"
Oct 31 08:41:07.595865 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:07.595828    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:41:07.596344 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:07.595941    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:41:07.597302 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:07.597267    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:41:09.310806 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:41:09.310773    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:41:17.596590 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:17.596536    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:41:17.608872 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:17.608842    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:41:27.595919 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:27.595879    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:41:37.596380 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:37.596330    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:41:39.316389 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:41:39.316351    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:41:47.596393 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:47.596305    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:41:57.595976 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:41:57.595924    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:42:07.596345 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:42:07.596301    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:42:09.310769 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:42:09.310728    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:42:17.595752 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:42:17.595702    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:42:27.596055 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:42:27.596003    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:42:37.596250 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:42:37.596199    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8001/health\": dial tcp 10.133.0.57:8001: connect: connection refused"
Oct 31 08:42:39.311243 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:42:39.311205    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:42:47.605136 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:42:47.605099    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:42:47.624529 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:42:47.624503    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:43:09.311735 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:43:09.311695    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:43:39.310173 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:43:39.310094    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:43:42.852502 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:42.852453    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"]
Oct 31 08:43:57.878201 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:57.878147    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main" containerID="cri-o://f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac" gracePeriod=45
Oct 31 08:43:58.494057 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.494012    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="llm-d-routing-sidecar" containerID="cri-o://242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c" gracePeriod=45
Oct 31 08:43:58.728676 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.728653    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:43:58.864682 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864657    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-tmp-dir\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.864846 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864690    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3446fda8-34bc-4ed4-859a-582906a74eff-kube-api-access-gj947\" (UniqueName: \"kubernetes.io/projected/3446fda8-34bc-4ed4-859a-582906a74eff-kube-api-access-gj947\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.864846 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864719    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-home\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-home\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.864846 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864745    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-dshm\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.864846 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864785    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3446fda8-34bc-4ed4-859a-582906a74eff-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3446fda8-34bc-4ed4-859a-582906a74eff-cabundle-cert\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.864846 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864801    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-model-cache\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.865119 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864861    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-kserve-provision-location\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.865119 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.864895    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3446fda8-34bc-4ed4-859a-582906a74eff-tls-certs\" (UniqueName: \"kubernetes.io/secret/3446fda8-34bc-4ed4-859a-582906a74eff-tls-certs\") pod \"3446fda8-34bc-4ed4-859a-582906a74eff\" (UID: \"3446fda8-34bc-4ed4-859a-582906a74eff\") "
Oct 31 08:43:58.865355 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.865321    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3446fda8-34bc-4ed4-859a-582906a74eff-cabundle-cert" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:43:58.865423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.865400    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-model-cache" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:43:58.865565 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.865533    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-home" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:43:58.867075 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.867039    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3446fda8-34bc-4ed4-859a-582906a74eff-kube-api-access-gj947" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "kube-api-access-gj947". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:43:58.867337 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.867309    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-dshm" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:43:58.867421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.867338    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3446fda8-34bc-4ed4-859a-582906a74eff-tls-certs" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:43:58.883597 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.883564    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-tmp-dir" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:43:58.895357 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.895333    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-kserve-provision-location" pod "3446fda8-34bc-4ed4-859a-582906a74eff" (UID: "3446fda8-34bc-4ed4-859a-582906a74eff"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:43:58.965560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965493    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:58.965560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965519    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3446fda8-34bc-4ed4-859a-582906a74eff-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:58.965560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965530    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:58.965560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965538    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj947\" (UniqueName: \"kubernetes.io/projected/3446fda8-34bc-4ed4-859a-582906a74eff-kube-api-access-gj947\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:58.965560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965548    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:58.965560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965555    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:58.965560 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965563    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3446fda8-34bc-4ed4-859a-582906a74eff-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:58.965862 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:58.965571    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3446fda8-34bc-4ed4-859a-582906a74eff-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:43:59.090159 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.090124    4532 generic.go:356] "Generic (PLEG): container finished" podID="3446fda8-34bc-4ed4-859a-582906a74eff" containerID="f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac" exitCode=0
Oct 31 08:43:59.090159 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.090153    4532 generic.go:356] "Generic (PLEG): container finished" podID="3446fda8-34bc-4ed4-859a-582906a74eff" containerID="242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c" exitCode=0
Oct 31 08:43:59.090332 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.090180    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"
Oct 31 08:43:59.090332 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.090179    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerDied","Data":"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"}
Oct 31 08:43:59.090332 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.090308    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerDied","Data":"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"}
Oct 31 08:43:59.090332 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.090329    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb" event={"ID":"3446fda8-34bc-4ed4-859a-582906a74eff","Type":"ContainerDied","Data":"aa89f6db95156feefda157740374cf319ec1a650d258b44eb4bd0e498e6ce78d"}
Oct 31 08:43:59.090507 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.090351    4532 scope.go:122] "RemoveContainer" containerID="f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"
Oct 31 08:43:59.098765 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.098743    4532 scope.go:122] "RemoveContainer" containerID="79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"
Oct 31 08:43:59.106049 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.106031    4532 scope.go:122] "RemoveContainer" containerID="242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"
Oct 31 08:43:59.112646 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.112623    4532 scope.go:122] "RemoveContainer" containerID="f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"
Oct 31 08:43:59.112887 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:43:59.112868    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac\": container with ID starting with f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac not found: ID does not exist" containerID="f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"
Oct 31 08:43:59.112952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.112897    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"} err="failed to get container status \"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac\": rpc error: code = NotFound desc = could not find container \"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac\": container with ID starting with f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac not found: ID does not exist"
Oct 31 08:43:59.112952 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.112915    4532 scope.go:122] "RemoveContainer" containerID="79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"
Oct 31 08:43:59.113128 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:43:59.113113    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3\": container with ID starting with 79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3 not found: ID does not exist" containerID="79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"
Oct 31 08:43:59.113168 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113130    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"} err="failed to get container status \"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3\": rpc error: code = NotFound desc = could not find container \"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3\": container with ID starting with 79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3 not found: ID does not exist"
Oct 31 08:43:59.113168 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113141    4532 scope.go:122] "RemoveContainer" containerID="242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"
Oct 31 08:43:59.113330 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:43:59.113317    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c\": container with ID starting with 242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c not found: ID does not exist" containerID="242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"
Oct 31 08:43:59.113372 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113332    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"} err="failed to get container status \"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c\": rpc error: code = NotFound desc = could not find container \"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c\": container with ID starting with 242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c not found: ID does not exist"
Oct 31 08:43:59.113372 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113344    4532 scope.go:122] "RemoveContainer" containerID="f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"
Oct 31 08:43:59.113583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113565    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac"} err="failed to get container status \"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac\": rpc error: code = NotFound desc = could not find container \"f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac\": container with ID starting with f6fd2a1de90acd6ead51a3cc6cfd0eb7e48f36b1ea1ae4d61699c49bfa4fa7ac not found: ID does not exist"
Oct 31 08:43:59.113633 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113583    4532 scope.go:122] "RemoveContainer" containerID="79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"
Oct 31 08:43:59.113813 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113795    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3"} err="failed to get container status \"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3\": rpc error: code = NotFound desc = could not find container \"79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3\": container with ID starting with 79c0aab0e1512a694b00fec44d389a81f042170e00e9108975b51c2d4f6fe8b3 not found: ID does not exist"
Oct 31 08:43:59.113869 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113813    4532 scope.go:122] "RemoveContainer" containerID="242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"
Oct 31 08:43:59.113998 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.113978    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c"} err="failed to get container status \"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c\": rpc error: code = NotFound desc = could not find container \"242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c\": container with ID starting with 242b31015ae55fc5cc95e9908d1474f42f1d7d6731c28209e88357a89070142c not found: ID does not exist"
Oct 31 08:43:59.253872 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.253816    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"]
Oct 31 08:43:59.256973 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.256948    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-2wwtb"]
Oct 31 08:43:59.623545 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:43:59.623520    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" path="/var/lib/kubelet/pods/3446fda8-34bc-4ed4-859a-582906a74eff/volumes"
Oct 31 08:44:09.309998 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:44:09.309963    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:44:17.874399 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874365    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"]
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874657    4532 cpu_manager.go:414] "containerMap: removing container" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="llm-d-routing-sidecar"
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874667    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="llm-d-routing-sidecar"
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874689    4532 cpu_manager.go:414] "containerMap: removing container" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main"
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874693    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main"
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874708    4532 cpu_manager.go:414] "containerMap: removing container" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="storage-initializer"
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874712    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="storage-initializer"
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874761    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="llm-d-routing-sidecar"
Oct 31 08:44:17.874868 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.874770    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="3446fda8-34bc-4ed4-859a-582906a74eff" containerName="main"
Oct 31 08:44:17.878878 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.878856    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.931016 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.930993    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"kube-root-ca.crt\""
Oct 31 08:44:17.959121 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.959088    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"openshift-service-ca.crt\""
Oct 31 08:44:17.959121 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.959101    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"openshift-gw-ca-root-cert\""
Oct 31 08:44:17.959264 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.959156    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"model-routing-only-gateway-openshift-default-dockercfg-ctrcs\""
Oct 31 08:44:17.981812 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.981785    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"]
Oct 31 08:44:17.997921 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.997899    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-credential-socket\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998010 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.997929    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27b8437-b693-4b1f-a154-4ad83b226313-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998010 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.997951    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4q5c\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-kube-api-access-p4q5c\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998084 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.998019    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998154 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.998139    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-socket\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998196 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.998161    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-data\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998242 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.998199    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-istio-token\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998320 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.998302    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27b8437-b693-4b1f-a154-4ad83b226313-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:17.998393 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:17.998376    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-certs\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099536 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099508    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27b8437-b693-4b1f-a154-4ad83b226313-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099544    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-certs\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099572    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-credential-socket\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099602    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27b8437-b693-4b1f-a154-4ad83b226313-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099634    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4q5c\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-kube-api-access-p4q5c\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099670    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099735    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-socket\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099763    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-data\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.099871 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099823    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-istio-token\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.100070 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.099966    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-certs\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.100194 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.100167    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-socket\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.100194 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.100186    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-credential-socket\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.100336 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.100215    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-data\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.100336 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.100325    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27b8437-b693-4b1f-a154-4ad83b226313-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.102475 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.102441    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.102633 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.102618    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27b8437-b693-4b1f-a154-4ad83b226313-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.156661 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.156599    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-istio-token\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.156861 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.156844    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4q5c\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-kube-api-access-p4q5c\") pod \"model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.191515 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.191491    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:18.612276 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.612244    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"]
Oct 31 08:44:18.615422 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:44:18.615397    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27b8437_b693_4b1f_a154_4ad83b226313.slice/crio-5bc88c1f8b74f57d0441919e70034e1f72b1e524e996230c6516865dd3c8817e WatchSource:0}: Error finding container 5bc88c1f8b74f57d0441919e70034e1f72b1e524e996230c6516865dd3c8817e: Status 404 returned error can't find the container with id 5bc88c1f8b74f57d0441919e70034e1f72b1e524e996230c6516865dd3c8817e
Oct 31 08:44:18.617371 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.617340    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:44:18.617484 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.617403    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:44:18.617484 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:18.617431    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:44:19.163947 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:19.163915    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" event={"ID":"e27b8437-b693-4b1f-a154-4ad83b226313","Type":"ContainerStarted","Data":"dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1"}
Oct 31 08:44:19.164313 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:19.163964    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" event={"ID":"e27b8437-b693-4b1f-a154-4ad83b226313","Type":"ContainerStarted","Data":"5bc88c1f8b74f57d0441919e70034e1f72b1e524e996230c6516865dd3c8817e"}
Oct 31 08:44:19.191941 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:19.191913    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:19.196369 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:19.196346    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:19.305222 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:19.304893    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" podStartSLOduration=2.304879189 podStartE2EDuration="2.304879189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:44:19.303971436 +0000 UTC m=+4509.806125226" watchObservedRunningTime="2026-10-31 08:44:19.304879189 +0000 UTC m=+4509.807032976"
Oct 31 08:44:20.168037 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:20.168006    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:20.168894 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:20.168872    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:39.239292 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:39.239260    4532 generic.go:356] "Generic (PLEG): container finished" podID="e27b8437-b693-4b1f-a154-4ad83b226313" containerID="dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1" exitCode=0
Oct 31 08:44:39.239722 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:39.239336    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" event={"ID":"e27b8437-b693-4b1f-a154-4ad83b226313","Type":"ContainerDied","Data":"dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1"}
Oct 31 08:44:39.239768 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:39.239726    4532 scope.go:122] "RemoveContainer" containerID="dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1"
Oct 31 08:44:39.240677 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:39.240648    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:44:39.240789 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:39.240730    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:44:39.240789 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:39.240779    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:44:39.311336 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:44:39.311309    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:44:40.244984 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:40.244948    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" event={"ID":"e27b8437-b693-4b1f-a154-4ad83b226313","Type":"ContainerStarted","Data":"885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1"}
Oct 31 08:44:41.192383 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:41.192346    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:41.192588 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:41.192567    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 31 08:44:42.191920 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:42.191881    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 31 08:44:43.192834 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:43.192793    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 31 08:44:44.192300 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:44.192260    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 31 08:44:45.195047 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:45.195016    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:45.195433 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:45.195382    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:44:45.196044 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:44:45.196027    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:45:09.310174 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:45:09.310133    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:45:39.312249 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:45:39.312214    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:45:40.122438 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.122405    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"]
Oct 31 08:45:40.125697 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.125681    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201268 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201236    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-certs\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201413 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201281    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gp5n\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-kube-api-access-4gp5n\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201501 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201402    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-credential-socket\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201501 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201484    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-data\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201509    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6449b464-08a9-4150-9203-4d4190117d1c-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201532    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-istio-token\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201621 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201597    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201812 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201635    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-socket\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.201812 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.201651    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6449b464-08a9-4150-9203-4d4190117d1c-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.218607 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.218587    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"openshift-gw-ca-root-cert\""
Oct 31 08:45:40.218714 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.218590    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"kube-root-ca.crt\""
Oct 31 08:45:40.218714 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.218590    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"openshift-service-ca.crt\""
Oct 31 08:45:40.228051 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.228034    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"model-routing-only-gateway-openshift-default-dockercfg-z44ln\""
Oct 31 08:45:40.231296 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.231274    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"]
Oct 31 08:45:40.302694 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302665    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-certs\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.302847 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302701    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gp5n\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-kube-api-access-4gp5n\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.302847 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302738    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-credential-socket\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.302847 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302765    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-data\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.302847 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302787    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6449b464-08a9-4150-9203-4d4190117d1c-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.302847 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302818    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-istio-token\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.302847 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302846    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.303185 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.302886    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-socket\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.303185 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.303016    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6449b464-08a9-4150-9203-4d4190117d1c-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.303185 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.303172    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-certs\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.303313 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.303212    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-data\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.303313 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.303267    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-credential-socket\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.303435 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.303420    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-socket\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.303571 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.303551    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6449b464-08a9-4150-9203-4d4190117d1c-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.305245 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.305221    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.305510 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.305490    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6449b464-08a9-4150-9203-4d4190117d1c-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.360046 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.360025    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gp5n\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-kube-api-access-4gp5n\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.360591 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.360569    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-istio-token\") pod \"model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.438570 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.438515    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:40.860304 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.860189    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"]
Oct 31 08:45:40.862811 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:45:40.862785    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6449b464_08a9_4150_9203_4d4190117d1c.slice/crio-0c080aece5b014e5ca7af63a393e5d05230595d2920a7291906aacd07690e84a WatchSource:0}: Error finding container 0c080aece5b014e5ca7af63a393e5d05230595d2920a7291906aacd07690e84a: Status 404 returned error can't find the container with id 0c080aece5b014e5ca7af63a393e5d05230595d2920a7291906aacd07690e84a
Oct 31 08:45:40.864571 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.864538    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:45:40.864655 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.864600    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:45:40.864655 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:40.864629    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:45:41.459853 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:41.459818    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" event={"ID":"6449b464-08a9-4150-9203-4d4190117d1c","Type":"ContainerStarted","Data":"b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112"}
Oct 31 08:45:41.460242 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:41.459871    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" event={"ID":"6449b464-08a9-4150-9203-4d4190117d1c","Type":"ContainerStarted","Data":"0c080aece5b014e5ca7af63a393e5d05230595d2920a7291906aacd07690e84a"}
Oct 31 08:45:41.606859 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:41.606811    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podStartSLOduration=1.606798124 podStartE2EDuration="1.606798124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:45:41.6060431 +0000 UTC m=+4592.108196889" watchObservedRunningTime="2026-10-31 08:45:41.606798124 +0000 UTC m=+4592.108951913"
Oct 31 08:45:42.439433 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:42.439398    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:42.444167 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:42.444138    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:42.464355 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:42.464325    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:42.465116 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:42.465099    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:44.486508 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:44.486411    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"]
Oct 31 08:45:44.486927 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:44.486611    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy" containerID="cri-o://885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1" gracePeriod=30
Oct 31 08:45:45.197268 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:45.197226    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 08:45:49.745550 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.745526    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:45:49.894199 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894162    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-socket\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894199 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894205    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e27b8437-b693-4b1f-a154-4ad83b226313-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27b8437-b693-4b1f-a154-4ad83b226313-istio-podinfo\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894452 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894238    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-data\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894452 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894259    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-istio-token\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-istio-token\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894452 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894328    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-envoy\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894452 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894358    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-certs\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894452 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894418    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-credential-socket\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894480    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e27b8437-b693-4b1f-a154-4ad83b226313-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27b8437-b693-4b1f-a154-4ad83b226313-istiod-ca-cert\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894501    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-data" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:45:49.894740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894520    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-kube-api-access-p4q5c\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-kube-api-access-p4q5c\") pod \"e27b8437-b693-4b1f-a154-4ad83b226313\" (UID: \"e27b8437-b693-4b1f-a154-4ad83b226313\") "
Oct 31 08:45:49.894740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894505    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-socket" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:45:49.894740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894662    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-credential-socket" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:45:49.894740 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894706    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-certs" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:45:49.895014 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894798    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.895014 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894815    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.895014 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894832    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.895014 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.894846    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.895389 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.895348    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e27b8437-b693-4b1f-a154-4ad83b226313-istiod-ca-cert" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:45:49.896810 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.896782    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e27b8437-b693-4b1f-a154-4ad83b226313-istio-podinfo" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:45:49.896917 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.896818    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-envoy" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:45:49.896964 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.896953    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-istio-token" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:45:49.897083 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.897064    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-kube-api-access-p4q5c" pod "e27b8437-b693-4b1f-a154-4ad83b226313" (UID: "e27b8437-b693-4b1f-a154-4ad83b226313"). InnerVolumeSpecName "kube-api-access-p4q5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:45:49.996167 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.996124    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e27b8437-b693-4b1f-a154-4ad83b226313-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.996167 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.996168    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e27b8437-b693-4b1f-a154-4ad83b226313-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.996418 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.996189    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4q5c\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-kube-api-access-p4q5c\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.996418 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.996205    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e27b8437-b693-4b1f-a154-4ad83b226313-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:49.996418 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:49.996222    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e27b8437-b693-4b1f-a154-4ad83b226313-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:45:50.497418 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.497382    4532 generic.go:356] "Generic (PLEG): container finished" podID="e27b8437-b693-4b1f-a154-4ad83b226313" containerID="885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1" exitCode=0
Oct 31 08:45:50.497613 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.497439    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"
Oct 31 08:45:50.497613 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.497437    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" event={"ID":"e27b8437-b693-4b1f-a154-4ad83b226313","Type":"ContainerDied","Data":"885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1"}
Oct 31 08:45:50.497613 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.497569    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz" event={"ID":"e27b8437-b693-4b1f-a154-4ad83b226313","Type":"ContainerDied","Data":"5bc88c1f8b74f57d0441919e70034e1f72b1e524e996230c6516865dd3c8817e"}
Oct 31 08:45:50.497613 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.497599    4532 scope.go:122] "RemoveContainer" containerID="885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1"
Oct 31 08:45:50.506713 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.506624    4532 scope.go:122] "RemoveContainer" containerID="dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1"
Oct 31 08:45:50.514142 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.514124    4532 scope.go:122] "RemoveContainer" containerID="885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1"
Oct 31 08:45:50.514397 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:45:50.514381    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1\": container with ID starting with 885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1 not found: ID does not exist" containerID="885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1"
Oct 31 08:45:50.514444 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.514405    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1"} err="failed to get container status \"885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1\": rpc error: code = NotFound desc = could not find container \"885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1\": container with ID starting with 885d56eecff7d2944d207cb7d2137ccf9fdcce88d60c9f613237707a280ec0f1 not found: ID does not exist"
Oct 31 08:45:50.514444 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.514422    4532 scope.go:122] "RemoveContainer" containerID="dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1"
Oct 31 08:45:50.514671 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:45:50.514651    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1\": container with ID starting with dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1 not found: ID does not exist" containerID="dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1"
Oct 31 08:45:50.514713 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.514680    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1"} err="failed to get container status \"dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1\": rpc error: code = NotFound desc = could not find container \"dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1\": container with ID starting with dee60b1990ff8d065bc1dae4dc50ac6db88ebd107be98f458e0b61a3244ae3c1 not found: ID does not exist"
Oct 31 08:45:50.711398 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.711362    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"]
Oct 31 08:45:50.776543 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:50.761001    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-gateway-openshift-default-5f6b699d5c-hlhhz"]
Oct 31 08:45:51.623318 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:51.623285    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" path="/var/lib/kubelet/pods/e27b8437-b693-4b1f-a154-4ad83b226313/volumes"
Oct 31 08:45:57.466020 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:57.465979    4532 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 08:45:57.526583 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:57.526554    4532 generic.go:356] "Generic (PLEG): container finished" podID="6449b464-08a9-4150-9203-4d4190117d1c" containerID="b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112" exitCode=0
Oct 31 08:45:57.526700 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:57.526613    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" event={"ID":"6449b464-08a9-4150-9203-4d4190117d1c","Type":"ContainerDied","Data":"b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112"}
Oct 31 08:45:57.526938 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:57.526914    4532 scope.go:122] "RemoveContainer" containerID="b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112"
Oct 31 08:45:57.528032 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:57.528005    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:45:57.528108 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:57.528070    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:45:57.528108 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:57.528101    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:45:58.532269 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:58.532227    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" event={"ID":"6449b464-08a9-4150-9203-4d4190117d1c","Type":"ContainerStarted","Data":"7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7"}
Oct 31 08:45:59.438864 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:59.438830    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:45:59.439059 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:45:59.439016    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 08:46:00.439205 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:00.439161    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 08:46:01.439594 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:01.439551    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 08:46:02.439956 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:02.439911    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 08:46:03.439161 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:03.439107    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 08:46:04.443684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:04.443650    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:46:04.444173 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:04.444038    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:46:04.444615 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:04.444592    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:46:09.310867 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:46:09.310830    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:46:39.310515 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:46:39.310458    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:46:40.142748 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.142716    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"]
Oct 31 08:46:40.143019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.143006    4532 cpu_manager.go:414] "containerMap: removing container" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy"
Oct 31 08:46:40.143065 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.143019    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy"
Oct 31 08:46:40.143065 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.143032    4532 cpu_manager.go:414] "containerMap: removing container" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy"
Oct 31 08:46:40.143065 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.143037    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy"
Oct 31 08:46:40.143187 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.143096    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy"
Oct 31 08:46:40.143225 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.143218    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="e27b8437-b693-4b1f-a154-4ad83b226313" containerName="istio-proxy"
Oct 31 08:46:40.146128 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.146112    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.180147 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.180113    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"]
Oct 31 08:46:40.228727 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.228696    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"odh-kserve-custom-ca-bundle\""
Oct 31 08:46:40.228914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.228727    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"seaweedfs-s3-creds\""
Oct 31 08:46:40.228914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.228741    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\""
Oct 31 08:46:40.228914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.228835    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\""
Oct 31 08:46:40.228914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.228876    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\""
Oct 31 08:46:40.228914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.228905    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-9dmmz\""
Oct 31 08:46:40.279410 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279385    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-tmp-dir\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.279531 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279415    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16ae7904-ca47-493a-b984-2cb113952fcd-cabundle-cert\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.279531 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279439    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-model-cache\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.279531 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279454    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dnpw\" (UniqueName: \"kubernetes.io/projected/16ae7904-ca47-493a-b984-2cb113952fcd-kube-api-access-2dnpw\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.279683 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279575    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16ae7904-ca47-493a-b984-2cb113952fcd-tls-certs\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.279683 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279621    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-home\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.279683 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279636    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-dshm\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.279683 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.279651    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380409    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-home\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380436    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-dshm\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380453    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380514    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-tmp-dir\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380654    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16ae7904-ca47-493a-b984-2cb113952fcd-cabundle-cert\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380706    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-model-cache\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380736    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dnpw\" (UniqueName: \"kubernetes.io/projected/16ae7904-ca47-493a-b984-2cb113952fcd-kube-api-access-2dnpw\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.380814 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380780    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16ae7904-ca47-493a-b984-2cb113952fcd-tls-certs\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.381181 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380893    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-tmp-dir\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.381181 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.380949    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-home\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.381181 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.381018    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.381348 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.381297    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16ae7904-ca47-493a-b984-2cb113952fcd-cabundle-cert\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.381406 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.381381    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-model-cache\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.382838 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.382820    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-dshm\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.383111 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.383094    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16ae7904-ca47-493a-b984-2cb113952fcd-tls-certs\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.442391 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.442335    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dnpw\" (UniqueName: \"kubernetes.io/projected/16ae7904-ca47-493a-b984-2cb113952fcd-kube-api-access-2dnpw\") pod \"prestop-hook-test-kserve-5f8f955bf6-kkpsc\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.460303 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.460279    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:40.881725 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:40.881697    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"]
Oct 31 08:46:40.884939 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:46:40.884914    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ae7904_ca47_493a_b984_2cb113952fcd.slice/crio-aa80189f28138694f8466c65771f7c9a2c51bd4524d2eae2a7494a2a3acfb180 WatchSource:0}: Error finding container aa80189f28138694f8466c65771f7c9a2c51bd4524d2eae2a7494a2a3acfb180: Status 404 returned error can't find the container with id aa80189f28138694f8466c65771f7c9a2c51bd4524d2eae2a7494a2a3acfb180
Oct 31 08:46:41.687698 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:41.687663    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" event={"ID":"16ae7904-ca47-493a-b984-2cb113952fcd","Type":"ContainerStarted","Data":"3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394"}
Oct 31 08:46:41.688080 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:41.687715    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" event={"ID":"16ae7904-ca47-493a-b984-2cb113952fcd","Type":"ContainerStarted","Data":"aa80189f28138694f8466c65771f7c9a2c51bd4524d2eae2a7494a2a3acfb180"}
Oct 31 08:46:43.697628 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:43.697598    4532 generic.go:356] "Generic (PLEG): container finished" podID="16ae7904-ca47-493a-b984-2cb113952fcd" containerID="3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394" exitCode=0
Oct 31 08:46:43.697994 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:43.697657    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" event={"ID":"16ae7904-ca47-493a-b984-2cb113952fcd","Type":"ContainerDied","Data":"3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394"}
Oct 31 08:46:44.704537 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:44.704493    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" event={"ID":"16ae7904-ca47-493a-b984-2cb113952fcd","Type":"ContainerStarted","Data":"a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed"}
Oct 31 08:46:44.925926 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:44.925864    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" podStartSLOduration=2.7812312390000002 podStartE2EDuration="4.925847955s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.144616716s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:46:44.925388958 +0000 UTC m=+4655.427542748" watchObservedRunningTime="2026-10-31 08:46:44.925847955 +0000 UTC m=+4655.428001748"
Oct 31 08:46:45.461049 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:45.461011    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:50.460833 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:50.460792    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:46:50.462168 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:46:50.462142    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Oct 31 08:47:00.461217 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:00.461170    4532 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": dial tcp 10.133.0.60:8000: connect: connection refused"
Oct 31 08:47:00.973276 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:00.973243    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"]
Oct 31 08:47:05.477407 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:05.477370    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"]
Oct 31 08:47:05.477815 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:05.477607    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy" containerID="cri-o://7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7" gracePeriod=30
Oct 31 08:47:09.311417 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:47:09.311377    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:47:10.725398 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.725375    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:47:10.810745 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.810719    4532 generic.go:356] "Generic (PLEG): container finished" podID="6449b464-08a9-4150-9203-4d4190117d1c" containerID="7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7" exitCode=0
Oct 31 08:47:10.810884 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.810785    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"
Oct 31 08:47:10.810884 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.810801    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" event={"ID":"6449b464-08a9-4150-9203-4d4190117d1c","Type":"ContainerDied","Data":"7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7"}
Oct 31 08:47:10.810884 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.810859    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq" event={"ID":"6449b464-08a9-4150-9203-4d4190117d1c","Type":"ContainerDied","Data":"0c080aece5b014e5ca7af63a393e5d05230595d2920a7291906aacd07690e84a"}
Oct 31 08:47:10.811066 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.810883    4532 scope.go:122] "RemoveContainer" containerID="7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7"
Oct 31 08:47:10.820163 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.820146    4532 scope.go:122] "RemoveContainer" containerID="b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112"
Oct 31 08:47:10.827728 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.827701    4532 scope.go:122] "RemoveContainer" containerID="7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7"
Oct 31 08:47:10.828015 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:47:10.827991    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7\": container with ID starting with 7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7 not found: ID does not exist" containerID="7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7"
Oct 31 08:47:10.828066 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.828028    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7"} err="failed to get container status \"7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7\": rpc error: code = NotFound desc = could not find container \"7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7\": container with ID starting with 7ae35730d03b27408aa2f7f9707f317f4ce7d73f0c2b3764e65d5f51fce7e7b7 not found: ID does not exist"
Oct 31 08:47:10.828066 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.828051    4532 scope.go:122] "RemoveContainer" containerID="b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112"
Oct 31 08:47:10.828322 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:47:10.828301    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112\": container with ID starting with b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112 not found: ID does not exist" containerID="b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112"
Oct 31 08:47:10.828398 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.828332    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112"} err="failed to get container status \"b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112\": rpc error: code = NotFound desc = could not find container \"b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112\": container with ID starting with b3d6a983e89ff2eab9b6e15b5e62b8ed26a92742690c4eba15228e98131db112 not found: ID does not exist"
Oct 31 08:47:10.836549 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836532    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-kube-api-access-4gp5n\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-kube-api-access-4gp5n\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.836615 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836590    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-credential-socket\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.836671 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836620    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-envoy\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.836797 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836778    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-istio-token\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-istio-token\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.836860 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836812    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-certs\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.836915 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836867    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-data\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.836972 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836919    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6449b464-08a9-4150-9203-4d4190117d1c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6449b464-08a9-4150-9203-4d4190117d1c-istiod-ca-cert\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.836972 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836952    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-socket\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.837100 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837042    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6449b464-08a9-4150-9203-4d4190117d1c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6449b464-08a9-4150-9203-4d4190117d1c-istio-podinfo\") pod \"6449b464-08a9-4150-9203-4d4190117d1c\" (UID: \"6449b464-08a9-4150-9203-4d4190117d1c\") "
Oct 31 08:47:10.837155 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.836917    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-credential-socket" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:47:10.837155 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837087    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-certs" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:47:10.837363 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837330    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-socket" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:47:10.837450 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837351    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6449b464-08a9-4150-9203-4d4190117d1c-istiod-ca-cert" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:47:10.837540 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837509    4532 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-credential-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.837540 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837530    4532 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.837657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837546    4532 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6449b464-08a9-4150-9203-4d4190117d1c-istiod-ca-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.837657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837560    4532 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-workload-socket\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.837657 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.837625    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-data" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:47:10.838870 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.838843    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-envoy" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:47:10.838990 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.838955    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-kube-api-access-4gp5n" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "kube-api-access-4gp5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:47:10.839259 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.839241    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-istio-token" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:47:10.839336 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.839284    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6449b464-08a9-4150-9203-4d4190117d1c-istio-podinfo" pod "6449b464-08a9-4150-9203-4d4190117d1c" (UID: "6449b464-08a9-4150-9203-4d4190117d1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 08:47:10.938763 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.938727    4532 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6449b464-08a9-4150-9203-4d4190117d1c-istio-podinfo\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.938763 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.938751    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gp5n\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-kube-api-access-4gp5n\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.938763 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.938760    4532 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-envoy\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.938763 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.938768    4532 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6449b464-08a9-4150-9203-4d4190117d1c-istio-token\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:10.939066 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:10.938778    4532 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6449b464-08a9-4150-9203-4d4190117d1c-istio-data\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:47:11.230483 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:11.230435    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"]
Oct 31 08:47:11.239605 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:11.234540    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-6d64f447cd-4t8tq"]
Oct 31 08:47:11.623850 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:11.623805    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6449b464-08a9-4150-9203-4d4190117d1c" path="/var/lib/kubelet/pods/6449b464-08a9-4150-9203-4d4190117d1c/volumes"
Oct 31 08:47:15.997937 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:47:15.997902    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="main" containerID="cri-o://a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed" gracePeriod=45
Oct 31 08:47:39.310042 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:47:39.310006    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:48:01.634909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.634885    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f8f955bf6-kkpsc_16ae7904-ca47-493a-b984-2cb113952fcd/main/0.log"
Oct 31 08:48:01.637512 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.637493    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:48:01.748860 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.748834    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-kserve-provision-location\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.748963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.748880    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16ae7904-ca47-493a-b984-2cb113952fcd-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16ae7904-ca47-493a-b984-2cb113952fcd-cabundle-cert\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.748963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.748900    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16ae7904-ca47-493a-b984-2cb113952fcd-tls-certs\" (UniqueName: \"kubernetes.io/secret/16ae7904-ca47-493a-b984-2cb113952fcd-tls-certs\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.748963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.748925    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-home\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-home\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.748963 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.748946    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16ae7904-ca47-493a-b984-2cb113952fcd-kube-api-access-2dnpw\" (UniqueName: \"kubernetes.io/projected/16ae7904-ca47-493a-b984-2cb113952fcd-kube-api-access-2dnpw\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.749150 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.748997    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-model-cache\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.749150 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.749024    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-tmp-dir\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.749150 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.749077    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-dshm\") pod \"16ae7904-ca47-493a-b984-2cb113952fcd\" (UID: \"16ae7904-ca47-493a-b984-2cb113952fcd\") "
Oct 31 08:48:01.749383 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.749322    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-model-cache" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:01.749383 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.749218    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ae7904-ca47-493a-b984-2cb113952fcd-cabundle-cert" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 08:48:01.749538 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.749452    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-home" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:01.749815 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.749772    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-tmp-dir" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:01.751297 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.751272    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ae7904-ca47-493a-b984-2cb113952fcd-tls-certs" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:48:01.751719 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.751698    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ae7904-ca47-493a-b984-2cb113952fcd-kube-api-access-2dnpw" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "kube-api-access-2dnpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:48:01.751818 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.751799    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-dshm" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:01.770734 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.770682    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-kserve-provision-location" pod "16ae7904-ca47-493a-b984-2cb113952fcd" (UID: "16ae7904-ca47-493a-b984-2cb113952fcd"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850626    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850664    4532 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-kserve-provision-location\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850680    4532 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/16ae7904-ca47-493a-b984-2cb113952fcd-cabundle-cert\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850695    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16ae7904-ca47-493a-b984-2cb113952fcd-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850709    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850728    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dnpw\" (UniqueName: \"kubernetes.io/projected/16ae7904-ca47-493a-b984-2cb113952fcd-kube-api-access-2dnpw\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850741    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:01.855207 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:01.850754    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16ae7904-ca47-493a-b984-2cb113952fcd-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:02.018389 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.018359    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f8f955bf6-kkpsc_16ae7904-ca47-493a-b984-2cb113952fcd/main/0.log"
Oct 31 08:48:02.018762 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.018729    4532 generic.go:356] "Generic (PLEG): container finished" podID="16ae7904-ca47-493a-b984-2cb113952fcd" containerID="a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed" exitCode=137
Oct 31 08:48:02.018869 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.018785    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" event={"ID":"16ae7904-ca47-493a-b984-2cb113952fcd","Type":"ContainerDied","Data":"a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed"}
Oct 31 08:48:02.018869 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.018798    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"
Oct 31 08:48:02.018869 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.018865    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc" event={"ID":"16ae7904-ca47-493a-b984-2cb113952fcd","Type":"ContainerDied","Data":"aa80189f28138694f8466c65771f7c9a2c51bd4524d2eae2a7494a2a3acfb180"}
Oct 31 08:48:02.018983 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.018897    4532 scope.go:122] "RemoveContainer" containerID="a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed"
Oct 31 08:48:02.028029 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.028005    4532 scope.go:122] "RemoveContainer" containerID="3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394"
Oct 31 08:48:02.034951 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.034931    4532 scope.go:122] "RemoveContainer" containerID="a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed"
Oct 31 08:48:02.035214 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:48:02.035195    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed\": container with ID starting with a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed not found: ID does not exist" containerID="a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed"
Oct 31 08:48:02.035258 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.035223    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed"} err="failed to get container status \"a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed\": rpc error: code = NotFound desc = could not find container \"a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed\": container with ID starting with a4cddf8601cf95014da61337198c0b19e0e05e1bf6ceb39005d5b10bc1b50eed not found: ID does not exist"
Oct 31 08:48:02.035258 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.035241    4532 scope.go:122] "RemoveContainer" containerID="3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394"
Oct 31 08:48:02.035488 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:48:02.035454    4532 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394\": container with ID starting with 3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394 not found: ID does not exist" containerID="3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394"
Oct 31 08:48:02.035530 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.035497    4532 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394"} err="failed to get container status \"3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394\": rpc error: code = NotFound desc = could not find container \"3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394\": container with ID starting with 3354e3bc4993618be43b97927f2b442e3446ef9f1571771887ae8493425cc394 not found: ID does not exist"
Oct 31 08:48:02.184096 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.184072    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"]
Oct 31 08:48:02.190049 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:02.190032    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-kkpsc"]
Oct 31 08:48:03.623351 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:03.623321    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" path="/var/lib/kubelet/pods/16ae7904-ca47-493a-b984-2cb113952fcd/volumes"
Oct 31 08:48:09.311197 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:48:09.311158    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:48:27.765004 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.764970    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"]
Oct 31 08:48:27.765423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765323    4532 cpu_manager.go:414] "containerMap: removing container" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="storage-initializer"
Oct 31 08:48:27.765423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765338    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="storage-initializer"
Oct 31 08:48:27.765423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765353    4532 cpu_manager.go:414] "containerMap: removing container" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy"
Oct 31 08:48:27.765423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765358    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy"
Oct 31 08:48:27.765423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765368    4532 cpu_manager.go:414] "containerMap: removing container" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="main"
Oct 31 08:48:27.765423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765375    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="main"
Oct 31 08:48:27.765423 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765426    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy"
Oct 31 08:48:27.765669 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.765434    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="16ae7904-ca47-493a-b984-2cb113952fcd" containerName="main"
Oct 31 08:48:27.767903 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.767888    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.814431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.814405    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"]
Oct 31 08:48:27.814732 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.814717    4532 cpu_manager.go:414] "containerMap: removing container" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy"
Oct 31 08:48:27.814797 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.814732    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy"
Oct 31 08:48:27.814835 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.814805    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="6449b464-08a9-4150-9203-4d4190117d1c" containerName="istio-proxy"
Oct 31 08:48:27.817474 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.817450    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.826437 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.826420    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 08:48:27.826700 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.826684    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 08:48:27.828971 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.828949    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829062 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.828985    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60fa182d-3749-4946-97f5-5381bc38e068-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829062 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.829012    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.829100    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829134 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.829118    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.829134    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjwnp\" (UniqueName: \"kubernetes.io/projected/60fa182d-3749-4946-97f5-5381bc38e068-kube-api-access-xjwnp\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.829154    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60fa182d-3749-4946-97f5-5381bc38e068-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829236 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.829177    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60fa182d-3749-4946-97f5-5381bc38e068-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.829324 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.829287    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.862741 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.862714    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 08:48:27.862838 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.862806    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-z29g8\""
Oct 31 08:48:27.871222 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.871200    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"]
Oct 31 08:48:27.919133 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.919106    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"]
Oct 31 08:48:27.929701 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.929680    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.929802 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.929715    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.929802 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.929732    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.929894 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.929796    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvdq\" (UniqueName: \"kubernetes.io/projected/ae968002-6186-44fd-af7f-83ed09d07a13-kube-api-access-zpvdq\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.929894 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.929858    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.929999 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.929972    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae968002-6186-44fd-af7f-83ed09d07a13-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.930056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930021    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae968002-6186-44fd-af7f-83ed09d07a13-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.930056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930038    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930125 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930098    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae968002-6186-44fd-af7f-83ed09d07a13-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.930125 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930106    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930189 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930159    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60fa182d-3749-4946-97f5-5381bc38e068-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930225 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930194    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930225 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930212    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.930297 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930239    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930297 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930264    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930370 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930306    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjwnp\" (UniqueName: \"kubernetes.io/projected/60fa182d-3749-4946-97f5-5381bc38e068-kube-api-access-xjwnp\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930370 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930359    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.930458 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930387    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60fa182d-3749-4946-97f5-5381bc38e068-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930458 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930410    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:27.930589 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930494    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60fa182d-3749-4946-97f5-5381bc38e068-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930589 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930579    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.930944 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.930922    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.931105 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.931074    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60fa182d-3749-4946-97f5-5381bc38e068-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.932554 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.932530    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60fa182d-3749-4946-97f5-5381bc38e068-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.933053 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.933033    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60fa182d-3749-4946-97f5-5381bc38e068-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.988067 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.988042    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjwnp\" (UniqueName: \"kubernetes.io/projected/60fa182d-3749-4946-97f5-5381bc38e068-kube-api-access-xjwnp\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:27.988195 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:27.988110    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60fa182d-3749-4946-97f5-5381bc38e068-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-6cswn\" (UID: \"60fa182d-3749-4946-97f5-5381bc38e068\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:28.031071 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031046    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031188 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031079    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031188 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031123    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031188 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031140    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031188 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031154    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvdq\" (UniqueName: \"kubernetes.io/projected/ae968002-6186-44fd-af7f-83ed09d07a13-kube-api-access-zpvdq\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031188 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031179    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae968002-6186-44fd-af7f-83ed09d07a13-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031398 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031239    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae968002-6186-44fd-af7f-83ed09d07a13-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031398 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031271    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae968002-6186-44fd-af7f-83ed09d07a13-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031398 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031314    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031575 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031457    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-workload-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031575 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031509    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-workload-certs\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031575 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031544    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-istio-data\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031878 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031848    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-credential-socket\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.031966 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.031949    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae968002-6186-44fd-af7f-83ed09d07a13-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.033200 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.033179    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae968002-6186-44fd-af7f-83ed09d07a13-istio-envoy\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.033573 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.033556    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae968002-6186-44fd-af7f-83ed09d07a13-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.079844 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.079819    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:28.088258 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.088230    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvdq\" (UniqueName: \"kubernetes.io/projected/ae968002-6186-44fd-af7f-83ed09d07a13-kube-api-access-zpvdq\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.088394 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.088363    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae968002-6186-44fd-af7f-83ed09d07a13-istio-token\") pod \"openshift-ai-inference-openshift-default-56f46b8cdd-827sh\" (UID: \"ae968002-6186-44fd-af7f-83ed09d07a13\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.128934 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.128812    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:28.500450 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.500384    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"]
Oct 31 08:48:28.502989 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:48:28.502962    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60fa182d_3749_4946_97f5_5381bc38e068.slice/crio-52379371f52e56e78d8af03a3ff8b8a66b6de966afef12810b206fa82b1de898 WatchSource:0}: Error finding container 52379371f52e56e78d8af03a3ff8b8a66b6de966afef12810b206fa82b1de898: Status 404 returned error can't find the container with id 52379371f52e56e78d8af03a3ff8b8a66b6de966afef12810b206fa82b1de898
Oct 31 08:48:28.505052 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.505024    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:48:28.505130 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.505088    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:48:28.505130 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.505121    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:48:28.554076 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.554054    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"]
Oct 31 08:48:28.555809 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:48:28.555777    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae968002_6186_44fd_af7f_83ed09d07a13.slice/crio-1b392552a3dca73b0e4600956b097dc418421a22efacf96ed20063c6032824fc WatchSource:0}: Error finding container 1b392552a3dca73b0e4600956b097dc418421a22efacf96ed20063c6032824fc: Status 404 returned error can't find the container with id 1b392552a3dca73b0e4600956b097dc418421a22efacf96ed20063c6032824fc
Oct 31 08:48:28.557711 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.557680    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:48:28.557819 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.557746    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:48:28.557819 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:28.557792    4532 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 31 08:48:29.120745 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.120712    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" event={"ID":"ae968002-6186-44fd-af7f-83ed09d07a13","Type":"ContainerStarted","Data":"c70726264846816f09118064021af2d171e2224d2d580534dd2ad80a212c203c"}
Oct 31 08:48:29.121386 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.121365    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" event={"ID":"ae968002-6186-44fd-af7f-83ed09d07a13","Type":"ContainerStarted","Data":"1b392552a3dca73b0e4600956b097dc418421a22efacf96ed20063c6032824fc"}
Oct 31 08:48:29.122747 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.122300    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" event={"ID":"60fa182d-3749-4946-97f5-5381bc38e068","Type":"ContainerStarted","Data":"8145255574d8e0f361d0fb3f52794c26e486941b9cb6310ad12fc4b241ab64d2"}
Oct 31 08:48:29.122747 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.122337    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" event={"ID":"60fa182d-3749-4946-97f5-5381bc38e068","Type":"ContainerStarted","Data":"52379371f52e56e78d8af03a3ff8b8a66b6de966afef12810b206fa82b1de898"}
Oct 31 08:48:29.129762 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.129742    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:29.131213 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.131193    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-827sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 08:48:29.131305 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.131233    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" podUID="ae968002-6186-44fd-af7f-83ed09d07a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 08:48:29.244439 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:29.244273    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" podStartSLOduration=2.244239238 podStartE2EDuration="2.244239238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:48:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:48:29.243440125 +0000 UTC m=+4759.745593917" watchObservedRunningTime="2026-10-31 08:48:29.244239238 +0000 UTC m=+4759.746393031"
Oct 31 08:48:30.081579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:30.081532    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:30.093688 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:30.093656    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 08:48:30.093961 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:30.093920    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" podUID="60fa182d-3749-4946-97f5-5381bc38e068" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 08:48:30.130080 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:30.130054    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-827sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 08:48:30.130508 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:30.130485    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" podUID="ae968002-6186-44fd-af7f-83ed09d07a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 08:48:31.082491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:31.081872    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 08:48:31.082491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:31.081945    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" podUID="60fa182d-3749-4946-97f5-5381bc38e068" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 08:48:31.129859 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:31.129815    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-827sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 08:48:31.130026 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:31.129868    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" podUID="ae968002-6186-44fd-af7f-83ed09d07a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 08:48:32.081488 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:32.080871    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 08:48:32.081488 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:32.080933    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" podUID="60fa182d-3749-4946-97f5-5381bc38e068" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 08:48:32.129772 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:32.129733    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-827sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 08:48:32.129936 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:32.129793    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" podUID="ae968002-6186-44fd-af7f-83ed09d07a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 08:48:33.087491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:33.085892    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 08:48:33.087491 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:33.085955    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" podUID="60fa182d-3749-4946-97f5-5381bc38e068" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 08:48:33.129878 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:33.129837    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-827sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 08:48:33.130044 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:33.129893    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" podUID="ae968002-6186-44fd-af7f-83ed09d07a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 08:48:34.080715 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:34.080679    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 08:48:34.080994 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:34.080964    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" podUID="60fa182d-3749-4946-97f5-5381bc38e068" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 08:48:34.130030 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:34.130001    4532 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-56f46b8cdd-827sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 08:48:34.130485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:34.130440    4532 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh" podUID="ae968002-6186-44fd-af7f-83ed09d07a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 08:48:35.084159 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:35.084128    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:35.084421 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:35.084388    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:35.085019 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:35.084999    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn"
Oct 31 08:48:35.133562 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:35.133542    4532 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:35.133900 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:35.133785    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:35.134438 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:35.134423    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-827sh"
Oct 31 08:48:35.254032 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:35.253982    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-56f46b8cdd-6cswn" podStartSLOduration=8.253971394 podStartE2EDuration="8.253971394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:48:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:48:29.371778748 +0000 UTC m=+4759.873932538" watchObservedRunningTime="2026-10-31 08:48:35.253971394 +0000 UTC m=+4765.756125176"
Oct 31 08:48:39.310615 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:48:39.310578    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:48:49.281576 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.281542    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"]
Oct 31 08:48:49.284787 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.284771    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.386781 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.386754    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-xwjcc\""
Oct 31 08:48:49.386910 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.386866    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Oct 31 08:48:49.387048 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.387030    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Oct 31 08:48:49.387145 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.387132    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Oct 31 08:48:49.388215 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.388199    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"]
Oct 31 08:48:49.436545 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.436521    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.436663 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.436557    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.436663 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.436644    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e756ae16-6a65-401f-973b-4f45b355f729-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.436748 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.436679    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npgsh\" (UniqueName: \"kubernetes.io/projected/e756ae16-6a65-401f-973b-4f45b355f729-kube-api-access-npgsh\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.436792 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.436776    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.436832 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.436809    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.537974    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538002    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538028 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538021    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e756ae16-6a65-401f-973b-4f45b355f729-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538248 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538038    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npgsh\" (UniqueName: \"kubernetes.io/projected/e756ae16-6a65-401f-973b-4f45b355f729-kube-api-access-npgsh\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538248 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538091    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538248 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538117    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538475 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538448    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538482    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.538579 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.538542    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.540327 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.540307    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.540564 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.540545    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e756ae16-6a65-401f-973b-4f45b355f729-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.594832 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.594807    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npgsh\" (UniqueName: \"kubernetes.io/projected/e756ae16-6a65-401f-973b-4f45b355f729-kube-api-access-npgsh\") pod \"rolling-upgrade-test-kserve-78f4d666cb-2pjmk\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.597405 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.597387    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:49.976656 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:49.976576    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"]
Oct 31 08:48:49.980214 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:48:49.980185    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode756ae16_6a65_401f_973b_4f45b355f729.slice/crio-c975484806d1a7ab8c87233232c2b15364bdd7edcb2415f9828c1542bd91b589 WatchSource:0}: Error finding container c975484806d1a7ab8c87233232c2b15364bdd7edcb2415f9828c1542bd91b589: Status 404 returned error can't find the container with id c975484806d1a7ab8c87233232c2b15364bdd7edcb2415f9828c1542bd91b589
Oct 31 08:48:50.201617 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:50.201578    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" event={"ID":"e756ae16-6a65-401f-973b-4f45b355f729","Type":"ContainerStarted","Data":"f85eff617cd96ca1f4573076892f87e8a576216ef76b949fa08e3280a1373033"}
Oct 31 08:48:50.201762 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:50.201654    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" event={"ID":"e756ae16-6a65-401f-973b-4f45b355f729","Type":"ContainerStarted","Data":"c975484806d1a7ab8c87233232c2b15364bdd7edcb2415f9828c1542bd91b589"}
Oct 31 08:48:50.316623 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:50.316580    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" podStartSLOduration=1.316567137 podStartE2EDuration="1.316567137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:48:50.315563432 +0000 UTC m=+4780.817717222" watchObservedRunningTime="2026-10-31 08:48:50.316567137 +0000 UTC m=+4780.818720927"
Oct 31 08:48:50.597947 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:50.597879    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:55.904320 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:55.904275    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"]
Oct 31 08:48:55.914191 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:48:55.914156    4532 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="f85eff617cd96ca1f4573076892f87e8a576216ef76b949fa08e3280a1373033" cmd=["/bin/sleep","15"]
Oct 31 08:48:55.914325 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:55.914205    4532 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 31 08:48:55.914325 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:48:55.914217    4532 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" podUID="e756ae16-6a65-401f-973b-4f45b355f729" containerName="main" containerID="cri-o://f85eff617cd96ca1f4573076892f87e8a576216ef76b949fa08e3280a1373033"
Oct 31 08:48:55.914325 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:55.914249    4532 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" podUID="e756ae16-6a65-401f-973b-4f45b355f729" containerName="main" containerID="cri-o://f85eff617cd96ca1f4573076892f87e8a576216ef76b949fa08e3280a1373033" gracePeriod=60
Oct 31 08:48:56.225989 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.225959    4532 generic.go:356] "Generic (PLEG): container finished" podID="e756ae16-6a65-401f-973b-4f45b355f729" containerID="f85eff617cd96ca1f4573076892f87e8a576216ef76b949fa08e3280a1373033" exitCode=0
Oct 31 08:48:56.226132 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.226026    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" event={"ID":"e756ae16-6a65-401f-973b-4f45b355f729","Type":"ContainerDied","Data":"f85eff617cd96ca1f4573076892f87e8a576216ef76b949fa08e3280a1373033"}
Oct 31 08:48:56.250675 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.250651    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:56.390009 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.389981    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-model-cache\") pod \"e756ae16-6a65-401f-973b-4f45b355f729\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") "
Oct 31 08:48:56.390166 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390036    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e756ae16-6a65-401f-973b-4f45b355f729-tls-certs\" (UniqueName: \"kubernetes.io/secret/e756ae16-6a65-401f-973b-4f45b355f729-tls-certs\") pod \"e756ae16-6a65-401f-973b-4f45b355f729\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") "
Oct 31 08:48:56.390166 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390064    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-home\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-home\") pod \"e756ae16-6a65-401f-973b-4f45b355f729\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") "
Oct 31 08:48:56.390166 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390091    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-dshm\") pod \"e756ae16-6a65-401f-973b-4f45b355f729\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") "
Oct 31 08:48:56.390166 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390146    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-tmp-dir\") pod \"e756ae16-6a65-401f-973b-4f45b355f729\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") "
Oct 31 08:48:56.390394 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390192    4532 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e756ae16-6a65-401f-973b-4f45b355f729-kube-api-access-npgsh\" (UniqueName: \"kubernetes.io/projected/e756ae16-6a65-401f-973b-4f45b355f729-kube-api-access-npgsh\") pod \"e756ae16-6a65-401f-973b-4f45b355f729\" (UID: \"e756ae16-6a65-401f-973b-4f45b355f729\") "
Oct 31 08:48:56.390394 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390298    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-model-cache" pod "e756ae16-6a65-401f-973b-4f45b355f729" (UID: "e756ae16-6a65-401f-973b-4f45b355f729"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:56.390394 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390346    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-home" pod "e756ae16-6a65-401f-973b-4f45b355f729" (UID: "e756ae16-6a65-401f-973b-4f45b355f729"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:56.390581 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.390499    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-tmp-dir" pod "e756ae16-6a65-401f-973b-4f45b355f729" (UID: "e756ae16-6a65-401f-973b-4f45b355f729"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:56.392431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.392400    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-dshm" pod "e756ae16-6a65-401f-973b-4f45b355f729" (UID: "e756ae16-6a65-401f-973b-4f45b355f729"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 08:48:56.392431 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.392414    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e756ae16-6a65-401f-973b-4f45b355f729-kube-api-access-npgsh" pod "e756ae16-6a65-401f-973b-4f45b355f729" (UID: "e756ae16-6a65-401f-973b-4f45b355f729"). InnerVolumeSpecName "kube-api-access-npgsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 08:48:56.392626 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.392499    4532 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e756ae16-6a65-401f-973b-4f45b355f729-tls-certs" pod "e756ae16-6a65-401f-973b-4f45b355f729" (UID: "e756ae16-6a65-401f-973b-4f45b355f729"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 08:48:56.490989 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.490936    4532 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npgsh\" (UniqueName: \"kubernetes.io/projected/e756ae16-6a65-401f-973b-4f45b355f729-kube-api-access-npgsh\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:56.490989 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.490958    4532 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-model-cache\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:56.490989 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.490968    4532 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e756ae16-6a65-401f-973b-4f45b355f729-tls-certs\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:56.490989 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.490977    4532 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-home\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:56.490989 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.490985    4532 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-dshm\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:56.490989 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:56.490992    4532 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e756ae16-6a65-401f-973b-4f45b355f729-tmp-dir\") on node \"ip-10-0-133-50.us-west-1.compute.internal\" DevicePath \"\""
Oct 31 08:48:57.230410 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:57.230381    4532 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"
Oct 31 08:48:57.230889 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:57.230382    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk" event={"ID":"e756ae16-6a65-401f-973b-4f45b355f729","Type":"ContainerDied","Data":"c975484806d1a7ab8c87233232c2b15364bdd7edcb2415f9828c1542bd91b589"}
Oct 31 08:48:57.230889 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:57.230537    4532 scope.go:122] "RemoveContainer" containerID="f85eff617cd96ca1f4573076892f87e8a576216ef76b949fa08e3280a1373033"
Oct 31 08:48:57.392195 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:57.392162    4532 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"]
Oct 31 08:48:57.441909 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:57.441887    4532 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-2pjmk"]
Oct 31 08:48:57.624056 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:48:57.624026    4532 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e756ae16-6a65-401f-973b-4f45b355f729" path="/var/lib/kubelet/pods/e756ae16-6a65-401f-973b-4f45b355f729/volumes"
Oct 31 08:49:09.310131 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:49:09.310097    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:49:39.310752 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:49:39.310666    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:50:09.310328 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:50:09.310290    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:50:39.311060 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:50:39.311021    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:50:59.184246 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:50:59.184211    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-56f46b8cdd-6cswn_60fa182d-3749-4946-97f5-5381bc38e068/istio-proxy/0.log"
Oct 31 08:50:59.504912 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:50:59.504830    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-56f46b8cdd-827sh_ae968002-6186-44fd-af7f-83ed09d07a13/istio-proxy/0.log"
Oct 31 08:51:04.127842 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:04.127809    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-56f46b8cdd-6cswn_60fa182d-3749-4946-97f5-5381bc38e068/istio-proxy/0.log"
Oct 31 08:51:04.343049 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:04.343016    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-56f46b8cdd-827sh_ae968002-6186-44fd-af7f-83ed09d07a13/istio-proxy/0.log"
Oct 31 08:51:08.063887 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:08.063857    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557c8f8584-nxqhs_6a9dac1c-1c08-4cc4-84f9-b521f779b7ed/authorino/0.log"
Oct 31 08:51:08.333247 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:08.333179    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-66498fdf6c-qxqkz_2e339cac-5d87-48ed-ae4d-a41a58cdb012/manager/0.log"
Oct 31 08:51:09.310809 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:51:09.310771    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:51:23.367914 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:23.367883    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6qkpn_65d23093-9f6e-4857-90be-3504d74e6dde/global-pull-secret-syncer/0.log"
Oct 31 08:51:26.998501 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:26.998452    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zgfmt_dbdb63cc-bc6e-424c-a528-5232fdfc561c/konnectivity-agent/0.log"
Oct 31 08:51:27.380057 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:27.380024    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-50.us-west-1.compute.internal_0fc2efca0f9b45f2393e3897a932b4b4/haproxy/0.log"
Oct 31 08:51:33.560908 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:33.560875    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557c8f8584-nxqhs_6a9dac1c-1c08-4cc4-84f9-b521f779b7ed/authorino/0.log"
Oct 31 08:51:34.013042 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:34.012948    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-66498fdf6c-qxqkz_2e339cac-5d87-48ed-ae4d-a41a58cdb012/manager/0.log"
Oct 31 08:51:36.729052 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.729009    4532 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"]
Oct 31 08:51:36.729546 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.729454    4532 cpu_manager.go:414] "containerMap: removing container" podUID="e756ae16-6a65-401f-973b-4f45b355f729" containerName="main"
Oct 31 08:51:36.729546 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.729492    4532 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e756ae16-6a65-401f-973b-4f45b355f729" containerName="main"
Oct 31 08:51:36.729672 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.729605    4532 memory_manager.go:365] "RemoveStaleState removing state" podUID="e756ae16-6a65-401f-973b-4f45b355f729" containerName="main"
Oct 31 08:51:36.732672 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.732648    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.758475 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.758448    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-sys\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.758573 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.758504    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-lib-modules\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.758648 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.758629    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-proc\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.758687 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.758670    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-podres\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.758730 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.758698    4532 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4qpd\" (UniqueName: \"kubernetes.io/projected/12fe78fa-20dd-456c-a637-07971d3f5544-kube-api-access-b4qpd\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.784774 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.784753    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9rj77\"/\"kube-root-ca.crt\""
Oct 31 08:51:36.834503 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.834478    4532 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9rj77\"/\"default-dockercfg-wfbtx\""
Oct 31 08:51:36.836445 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.836418    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"]
Oct 31 08:51:36.859006 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.858983    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-sys\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859141 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859014    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-lib-modules\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859141 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859056    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-proc\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859141 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859097    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-podres\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859141 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859116    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-sys\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859303 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859147    4532 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4qpd\" (UniqueName: \"kubernetes.io/projected/12fe78fa-20dd-456c-a637-07971d3f5544-kube-api-access-b4qpd\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859303 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859150    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-lib-modules\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859303 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859189    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-proc\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.859303 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.859198    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/12fe78fa-20dd-456c-a637-07971d3f5544-podres\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:36.885806 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.885787    4532 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9rj77\"/\"openshift-service-ca.crt\""
Oct 31 08:51:36.915329 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:36.915305    4532 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4qpd\" (UniqueName: \"kubernetes.io/projected/12fe78fa-20dd-456c-a637-07971d3f5544-kube-api-access-b4qpd\") pod \"perf-node-gather-daemonset-v9tpg\" (UID: \"12fe78fa-20dd-456c-a637-07971d3f5544\") " pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:37.044610 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:37.044564    4532 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:37.408875 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:37.408789    4532 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"]
Oct 31 08:51:37.411772 ip-10-0-133-50 kubenswrapper[4532]: W1031 08:51:37.411739    4532 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod12fe78fa_20dd_456c_a637_07971d3f5544.slice/crio-2e430b2dadb3f36b2250ade3a8ef2fe795fffd00d89666cf8320cd77dce4a5a4 WatchSource:0}: Error finding container 2e430b2dadb3f36b2250ade3a8ef2fe795fffd00d89666cf8320cd77dce4a5a4: Status 404 returned error can't find the container with id 2e430b2dadb3f36b2250ade3a8ef2fe795fffd00d89666cf8320cd77dce4a5a4
Oct 31 08:51:37.845708 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:37.845679    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg" event={"ID":"12fe78fa-20dd-456c-a637-07971d3f5544","Type":"ContainerStarted","Data":"3a34cc52bf3bc8aa8fdbab44044c29472916274321a0c071aebf1642c56ce7c1"}
Oct 31 08:51:37.846091 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:37.845725    4532 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg" event={"ID":"12fe78fa-20dd-456c-a637-07971d3f5544","Type":"ContainerStarted","Data":"2e430b2dadb3f36b2250ade3a8ef2fe795fffd00d89666cf8320cd77dce4a5a4"}
Oct 31 08:51:37.846091 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:37.845776    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:37.957809 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:37.957754    4532 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg" podStartSLOduration=1.957739688 podStartE2EDuration="1.957739688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 08:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:51:37.957146359 +0000 UTC m=+4948.459300149" watchObservedRunningTime="2026-10-31 08:51:37.957739688 +0000 UTC m=+4948.459893474"
Oct 31 08:51:39.310821 ip-10-0-133-50 kubenswrapper[4532]: E1031 08:51:39.310787    4532 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5gj7z" containerName="extractor"
Oct 31 08:51:43.858998 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:43.858965    4532 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9rj77/perf-node-gather-daemonset-v9tpg"
Oct 31 08:51:46.496636 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:46.496592    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rp2rq_0755a07b-a0e0-4f53-b4aa-7013a5e2896f/node-exporter/0.log"
Oct 31 08:51:46.908487 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:46.908401    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rp2rq_0755a07b-a0e0-4f53-b4aa-7013a5e2896f/kube-rbac-proxy/0.log"
Oct 31 08:51:47.321395 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:47.321369    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rp2rq_0755a07b-a0e0-4f53-b4aa-7013a5e2896f/init-textfile/0.log"
Oct 31 08:51:49.004752 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:49.004718    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_edbce910-2fca-4c8d-b6bc-a83141840f04/prometheus/0.log"
Oct 31 08:51:49.405476 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:49.405444    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_edbce910-2fca-4c8d-b6bc-a83141840f04/config-reloader/0.log"
Oct 31 08:51:49.817129 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:49.817104    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_edbce910-2fca-4c8d-b6bc-a83141840f04/thanos-sidecar/0.log"
Oct 31 08:51:50.230803 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:50.230721    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_edbce910-2fca-4c8d-b6bc-a83141840f04/kube-rbac-proxy-web/0.log"
Oct 31 08:51:50.643101 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:50.643072    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_edbce910-2fca-4c8d-b6bc-a83141840f04/kube-rbac-proxy/0.log"
Oct 31 08:51:51.055738 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:51.055713    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_edbce910-2fca-4c8d-b6bc-a83141840f04/kube-rbac-proxy-thanos/0.log"
Oct 31 08:51:51.470485 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:51.470397    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_edbce910-2fca-4c8d-b6bc-a83141840f04/init-config-reloader/0.log"
Oct 31 08:51:54.378314 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:54.378279    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7876dcdfc5-dlqh7_ef7fe687-91a4-4957-9526-b25b78f23309/thanos-query/0.log"
Oct 31 08:51:54.789684 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:54.789654    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7876dcdfc5-dlqh7_ef7fe687-91a4-4957-9526-b25b78f23309/kube-rbac-proxy-web/0.log"
Oct 31 08:51:55.202078 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:55.202006    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7876dcdfc5-dlqh7_ef7fe687-91a4-4957-9526-b25b78f23309/kube-rbac-proxy/0.log"
Oct 31 08:51:55.613680 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:55.613648    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7876dcdfc5-dlqh7_ef7fe687-91a4-4957-9526-b25b78f23309/prom-label-proxy/0.log"
Oct 31 08:51:56.029142 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:56.029110    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7876dcdfc5-dlqh7_ef7fe687-91a4-4957-9526-b25b78f23309/kube-rbac-proxy-rules/0.log"
Oct 31 08:51:56.442327 ip-10-0-133-50 kubenswrapper[4532]: I1031 08:51:56.442252    4532 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7876dcdfc5-dlqh7_ef7fe687-91a4-4957-9526-b25b78f23309/kube-rbac-proxy-metrics/0.log"